-- Logs begin at Fri 2022-08-19 14:36:36 UTC, end at Fri 2022-08-19 15:40:59 UTC. -- Aug 19 15:08:33.741256 dv-cinder-835384-20-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 19 15:08:33 2022] *** Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: nodename: dv-cinder-835384-20-iscsi Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: machine: x86_64 Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: clock source: unix Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: pcre jit disabled Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: detected number of CPU cores: 4 Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: current working directory: / Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: detected binary path: /usr/bin/uwsgi-core Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: your processes number limit is 47680 Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: your memory page size is 4096 bytes Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: detected max file descriptor number: 2048 Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: lock engine: pthread robust mutexes Aug 19 15:08:33.752417 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: thunder lock: enabled Aug 19 15:08:33.753649 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 19 15:08:33.753649 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 19 15:08:33.793203 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: Python main interpreter initialized at 0x560c7b4de9c0 Aug 19 15:08:33.793203 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: python threads support enabled Aug 19 15:08:33.793203 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: your server socket listen backlog is limited to 100 connections Aug 19 15:08:33.793203 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: your mercy for graceful operations on workers is 90 seconds Aug 19 15:08:33.793593 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: mapped 403077 bytes (393 KB) for 2 cores Aug 19 15:08:33.793670 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: *** Operational MODE: preforking *** Aug 19 15:08:33.793898 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: *** uWSGI is running in multiple interpreter mode *** Aug 19 15:08:33.793898 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: spawned uWSGI master process (pid: 81490) Aug 19 15:08:33.794109 dv-cinder-835384-20-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 19 15:08:33.795213 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: spawned uWSGI worker 1 (pid: 81503, cores: 1) Aug 19 15:08:33.795213 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: spawned uWSGI worker 2 (pid: 81504, cores: 1) Aug 19 15:08:33.795213 dv-cinder-835384-20-iscsi devstack@keystone.service[81490]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 19 15:08:38.121321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 15:08:38.121321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: from cryptography.utils import int_from_bytes Aug 19 15:08:38.122128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 15:08:38.122128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: from cryptography.utils import int_from_bytes Aug 19 15:08:38.122250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 15:08:38.122250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Aug 19 15:08:38.122250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 15:08:38.122250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Aug 19 15:08:38.172897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 19 15:08:38.173083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 19 15:08:38.173267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] command line args: [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 19 15:08:38.173359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 19 15:08:38.173507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 19 15:08:38.173593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 19 15:08:38.173704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ================================================================================ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 19 15:08:38.173790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] command line args: [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 19 15:08:38.173959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.174044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 19 15:08:38.174142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ================================================================================ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 19 15:08:38.174230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_dir = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.174388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.174476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] admin_token = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.174569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_source = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.174653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config_dir = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.174756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] debug = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.174896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.174986 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.175193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config_source = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.175193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] debug = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.175310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.175383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.175473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.175679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.175770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.175832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.175991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_config_append = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_config_append = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176815 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_dir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.176939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.177082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_dir = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.177183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.177264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.177388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.177481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.177619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.177707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.177867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.177958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.178028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.178155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.178274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.178355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.178450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.178558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.178698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.178790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.178937 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.179034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.179184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.179278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.179407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.179495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.179644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.179734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.179880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.179958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.180128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.180208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.180376 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.180495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.180675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.180906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.75/identity {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.181091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] publish_errors = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.181173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.75/identity {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.181329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.181415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] publish_errors = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.181564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.181662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.181736 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.181807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.181969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.182052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.182756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.182858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.182987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.183100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] standard_threads = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.183223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] standard_threads = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.183386 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.183469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.183590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.183695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.183773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.183892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_journal = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_json = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_journal = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_stderr = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_json = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_syslog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_stderr = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_syslog = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 15:08:38.184875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.185063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.185197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.185312 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.185390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.185577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.185660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.185823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.185908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.186067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.186145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.186312 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.186394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.186555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.186639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.186799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.186881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.187047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.187124 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.187300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.187384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.187537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.187623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.187781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.187902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.188021 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.188106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.188266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.188343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.188542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.188621 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.188783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.188872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.189026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.189104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.189286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.189362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.189543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.189697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.189771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.189841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.190026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.190108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.190287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.190373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.190543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.190618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.190796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.190891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.191033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.191116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.191268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.191351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.191503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.191579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.191741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.191826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.191985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.192068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.192232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.192314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.192492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.192573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.192729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.192816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.192965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.193048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.193218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.193296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.193458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.193542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.193699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.193780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.193933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.194011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.194175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.194251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.194414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.194491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.194769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.194861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.194861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.195003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.195103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.195182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.195327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.195400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.195565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.195649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.195811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.195894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.196044 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.196122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.196286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.196368 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.196547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.196627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.196783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.196887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.197026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.197136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.197266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.197375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.197486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.197592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.197788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.198472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.198594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.198774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.198861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.198986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.199063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.199225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.199314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.199399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.199472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.199587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.199664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.199821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.199902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.200080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.200163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.200326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.200446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.200604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.200694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.200846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.200939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.201086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.201170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.201328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.201407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.201569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.201651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.201810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.201893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.202054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.202180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.202319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.202443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.202561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.202645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.202802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.202881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.203046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.203125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.203292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.203380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.203534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.203626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.203774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.203859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.204018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.204099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.204259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.204337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.204525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.204607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.204770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.204851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.205012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.205090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.205252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.205329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.205491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.205571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.205734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.205822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.205975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.206060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.206220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.206299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.206462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.206542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.206707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.206784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.206948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.207033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.207191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.207270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.207430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.207508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.207661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.207737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.207886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.207966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.208125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.208269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.208364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.208473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.208605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.208686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.208851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.208940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.209090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.209188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.209336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.209417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.209579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.209660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.209835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.209937 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.210077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.210161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.210328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.210402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.210567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.external = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.210664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.210803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.210882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.211056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.external = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.211133 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.211292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.211401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.211551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.password = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.211638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.211788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.token = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.211868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.212032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.212112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.password = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.212268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.token = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.212355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.212539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.212623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.212774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.212852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.213025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.213113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.213263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.213342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.213523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.213602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.213767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.213853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.214008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.214180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.214259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.214437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.214518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.214691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.214772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.214931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.215011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.215180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.215263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.215423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.215504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.215677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.215756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.215934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.216018 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.216177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.216263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.216445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.216529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.216708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.216788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.216975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.217056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.217243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.217513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.217594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.217764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.217853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.218027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.218106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.218269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.218348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.218510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.218597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.218763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.218841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.219013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.219093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.219249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.219335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.219501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.219580 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.219745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.219824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.220003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.220083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.220244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.220329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.220528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.220613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.220773 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.220855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.221025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.221103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.221267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.221346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.221517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.221597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.221769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.221848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.222011 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.222089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.222258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.222335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.222492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.222585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.222722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.222814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.222966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.223046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.223218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.223296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.223473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.223552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.223709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.223788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.223947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.224033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.224184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.224263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.224463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.224548 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.224719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.224799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.224973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.225054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.225233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.225316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.225484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.225564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.225758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.225845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.226003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.226085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.226243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.226324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.226479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.226558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.226722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.226801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.226953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.227033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.227193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.227273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.227438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.227518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.227680 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.227759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.227932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.228012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.228168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.228249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.228434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.228563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.228688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.228768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.228929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.229042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.229328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.229478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.229588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.229828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.229920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.230073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.230151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.230309 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.230388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.230545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.230625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.230790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.230870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.231033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.231119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.231279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.231360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.231521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.231607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.231763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.231842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.232016 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.232098 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.232254 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.232370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.232529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.232604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.232744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.232825 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.232998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.233078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.233241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.233319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.233482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.233562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.233723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.233802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.233963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.234042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.234197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.234277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.234443 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.234523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.234691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.234769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.234931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.235011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.235180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.235260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.235429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.235509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.235670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.235755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.235907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.235987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.236138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.236216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.236378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.236480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.236638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.236721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.236873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.236953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.237105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.237185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.237355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.237454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.237605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.237690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.237855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.238106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.238341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.238514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.238594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.238822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.239064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.239306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.239384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.239546 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.239647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.239796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.239889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.240049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.240129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.240292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.240370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.240581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.240677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.240820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.240900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.241073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.241152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.241316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.241404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.241559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.241639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.241801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.241879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.242052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.242139 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.242288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.242367 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.242525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.242603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.242765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.242852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.243005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.243085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.243245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.243325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.243507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.243670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.243750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.243823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.243964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.244043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.244203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.244291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.244470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.244557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.245278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.245361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.245526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.245619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.245774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.245854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.246018 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.246097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.246271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.246360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.246507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.246587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.246762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.246842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.247016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.247101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.247271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.247402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.247512 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.247592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.247762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.247843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.247995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.248075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.248247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.248327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.248520 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.248602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.248760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.248841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.249010 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.249090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.249261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.249342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.249510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.249589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.249758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.249837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.250010 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.250166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.250247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.250412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.250492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.250669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.250758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.250932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.251013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.251204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.251284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.251459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.251611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.251693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.251855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.251934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.252108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.252189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.252364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.252478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.252641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.252722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.252895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.252986 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.253134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.253212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.253389 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.253467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.253644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.253731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.253887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.253967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.254124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.254204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.254361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.254450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.254597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.254677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.254837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.254915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.255082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.255170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.255335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.255414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.255577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.255655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.255816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.255903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.256052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.256132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.256292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.256372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.256558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.256654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.256809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.256889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.257050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.257130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.257292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.257379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.257540 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.257620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.257779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.257857 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.258028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.258114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.258273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.258352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.258507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.258586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.258749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.258839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.258991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.259071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.259229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.259308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.259478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.259568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.259730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.259811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.259963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.260043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.260225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.260342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.261097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.261224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.261429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.261509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.261687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.261776 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.261934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.262014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.262189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.262271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.262435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.262515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.262674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.262754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.262917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.262999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.263157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.263237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.263397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.263478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.263647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.263728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.263882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.263961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.264130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.264209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.264373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.264516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.264596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 19 15:08:38.264670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.264890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.265141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.265377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.265625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.265867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 15:08:38.266041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 19 15:08:38.294246 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.294475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.294562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.294703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.294801 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.294960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.303045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.303334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.332305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.332562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.332649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.332887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.386443 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.386685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.436023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.436172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.436247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.436335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.594762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.595001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.595223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.595450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.595652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.595740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.595883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.595965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.596108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.596330 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.596644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.596800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.619516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.619643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.620434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.620618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.620704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.620909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.621049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.621185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.621320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.621447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.621447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.621678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.621798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.621798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.632670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.632956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.645593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.645775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.645933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.646102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.646227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.646351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.646473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.646598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.646731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.646860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.646983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.647121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.647250 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.647384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 15:08:38.664010 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.664220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 15:08:38.664348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.664588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.664713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 15:08:38.664849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 15:08:38.664972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.665096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.665223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 15:08:38.665362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.665500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.665680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.665803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.666055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.666184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.666976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.675919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.676207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.676504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.676613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.676798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.676931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.677060 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.677352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.684500 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.684917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.687363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.687650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.687805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.687963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.688103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.688227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.688375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.688522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.688690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.688787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.688982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.689063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.689251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.689333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.689532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.689678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.689815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.689962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.690214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.690341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.690495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.690776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.712542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.713172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.714138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.714422 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.714675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.714965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.719075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.719364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.719633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.719716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.719926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.720217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.726262 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.726552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.726633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.726840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.731350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.731447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.731640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.731736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.731927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.732014 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.732212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.732293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.732516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.732603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.732798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.732898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.733071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.733163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.733347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.733426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.751271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.751401 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.751551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.751659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.755707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.755819 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.758660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.758962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.759892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.760196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.764359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.766163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.767650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.769356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.771866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.772374 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.772625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.772901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.772998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.773211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.773292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.773673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.773761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.773845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.773921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.774066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.789755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.790046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.790724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.790820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.790820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.791350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.791350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.791350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.798131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.798534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.799660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.799936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.800091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.800340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.802586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.802875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.802999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.803214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.803357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.803504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.803637 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.803764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.803888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.804142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.815610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.815893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.816179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.816329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.816489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.816640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.816785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.816928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.817057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.817192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.817324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.817451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.817732 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.818019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.833493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.833670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.836543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.836677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.839465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.839714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.840949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.841169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.842427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.842638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.842765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.842930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.843056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.843227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.849462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.849633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.850925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.851073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.854464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.854727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.854873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.855055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.855203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.855343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.855489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.855702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.863208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.863506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.863642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.863804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.863935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.864111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.871239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.871524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.871659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 15:08:38.871801 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.871929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.872084 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.872223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.872352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.872504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.872658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.872790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.872941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.873082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.873216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.873358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.873488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.873617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.873758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.873887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.874018 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.874154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.874339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.874465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.874595 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.874758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.874885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.875022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.875149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 15:08:38.909152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560c7b4de9c0 pid: 81503 (default app) Aug 19 15:08:38.909275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560c7b4de9c0 pid: 81504 (default app) Aug 19 15:08:38.912548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe07e39-5590-4527-9c76-14b645030616 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:38.912839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe07e39-5590-4527-9c76-14b645030616 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:38.913114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe07e39-5590-4527-9c76-14b645030616 None None] PATH_INFO: `/v3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:38.917759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1/1] 10.222.0.75 () {50 vars in 791 bytes} [Fri Aug 19 15:08:38 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:45.606453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2952c-e18d-4603-88f4-7c1cca3a3b8a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:45.606984 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2952c-e18d-4603-88f4-7c1cca3a3b8a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:45.607130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2952c-e18d-4603-88f4-7c1cca3a3b8a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:45.611733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1/2] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:08:45.616962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:45.617373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:45.617513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:45.688339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: INFO dbcounter [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] Registered counter for database keystone Aug 19 15:08:45.707333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG oslo_db.sqlalchemy.engines [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81503) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 19 15:08:45.714105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writer thread running {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 19 15:08:45.892316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81503) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 19 15:08:45.892690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81503) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 19 15:08:45.948318 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:45.949984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:46.079669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 2/3] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 19 15:08:46.541854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2952c-e18d-4603-88f4-7c1cca3a3b8a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:46.611752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: INFO dbcounter [None req-7fd2952c-e18d-4603-88f4-7c1cca3a3b8a None None] Registered counter for database keystone Aug 19 15:08:46.633767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG oslo_db.sqlalchemy.engines [None req-7fd2952c-e18d-4603-88f4-7c1cca3a3b8a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81504) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 19 15:08:46.641091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writer thread running {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 19 15:08:46.762540 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2952c-e18d-4603-88f4-7c1cca3a3b8a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:46.766793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823239de-1529-4d89-8621-5ba8d6318e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:46.767407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823239de-1529-4d89-8621-5ba8d6318e05 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:46.767683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823239de-1529-4d89-8621-5ba8d6318e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:46.767951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823239de-1529-4d89-8621-5ba8d6318e05 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:46.776071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823239de-1529-4d89-8621-5ba8d6318e05 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:46.829980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WARNING keystone.common.rbac_enforcer.enforcer [None req-823239de-1529-4d89-8621-5ba8d6318e05 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 19 15:08:46.830247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823239de-1529-4d89-8621-5ba8d6318e05 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:46.841342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 2/4] 10.222.0.75 () {58 vars in 1324 bytes} [Fri Aug 19 15:08:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 15:08:46.847523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:46.862025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08a48-7855-4d97-a0d1-e32f4dde350e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:46.866603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b24356-d6e4-4727-baa1-82da77adad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:46.867700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b24356-d6e4-4727-baa1-82da77adad43 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:46.867987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b24356-d6e4-4727-baa1-82da77adad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:46.868264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b24356-d6e4-4727-baa1-82da77adad43 admin admin] PATH_INFO: `/v3/projects/a9a96491d392440daeaecef9eef8a152` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:46.869875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b24356-d6e4-4727-baa1-82da77adad43 admin admin] RBAC: Authorizing `identity:get_project(project_id=a9a96491d392440daeaecef9eef8a152)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:46.923365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WARNING keystone.common.rbac_enforcer.enforcer [None req-11b24356-d6e4-4727-baa1-82da77adad43 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 19 15:08:46.923716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b24356-d6e4-4727-baa1-82da77adad43 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:46.928354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 3/5] 10.222.0.75 () {56 vars in 1207 bytes} [Fri Aug 19 15:08:46 2022] GET /identity/v3/projects/a9a96491d392440daeaecef9eef8a152 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:48.801416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bff172-7d91-42cb-b6f6-940ceabfed8c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:48.801943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bff172-7d91-42cb-b6f6-940ceabfed8c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:48.801943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bff172-7d91-42cb-b6f6-940ceabfed8c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:48.806037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 3/6] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:08:48.812069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a55aa-abfa-4d6e-bdda-ecc352356bed None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:48.812475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a55aa-abfa-4d6e-bdda-ecc352356bed None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:48.812671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a55aa-abfa-4d6e-bdda-ecc352356bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:48.832835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-219a55aa-abfa-4d6e-bdda-ecc352356bed None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:48.834351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-219a55aa-abfa-4d6e-bdda-ecc352356bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:48.842578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 4/7] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 19 15:08:49.111063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bff172-7d91-42cb-b6f6-940ceabfed8c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:49.124147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bff172-7d91-42cb-b6f6-940ceabfed8c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:49.128805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e6c53-3efb-4a97-abe4-e19bd881c97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:49.129432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e6c53-3efb-4a97-abe4-e19bd881c97f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:49.129769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e6c53-3efb-4a97-abe4-e19bd881c97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:49.130044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e6c53-3efb-4a97-abe4-e19bd881c97f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:49.138218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e6c53-3efb-4a97-abe4-e19bd881c97f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:49.140038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e6c53-3efb-4a97-abe4-e19bd881c97f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:49.148648 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 4/8] 10.222.0.75 () {58 vars in 1324 bytes} [Fri Aug 19 15:08:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:08:49.154450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a55aa-abfa-4d6e-bdda-ecc352356bed None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:49.163551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a55aa-abfa-4d6e-bdda-ecc352356bed None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:49.168433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b503f73-3cee-4f29-bdad-ce8df04eb3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:49.169056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b503f73-3cee-4f29-bdad-ce8df04eb3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:49.169337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b503f73-3cee-4f29-bdad-ce8df04eb3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:49.169611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b503f73-3cee-4f29-bdad-ce8df04eb3e3 admin admin] PATH_INFO: `/v3/users/0ba95fb65871415ca38c574ddd6396b8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:49.170910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b503f73-3cee-4f29-bdad-ce8df04eb3e3 admin admin] RBAC: Authorizing `identity:get_user(user_id=0ba95fb65871415ca38c574ddd6396b8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:49.172789 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b503f73-3cee-4f29-bdad-ce8df04eb3e3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:49.174309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 5/9] 10.222.0.75 () {56 vars in 1198 bytes} [Fri Aug 19 15:08:49 2022] GET /identity/v3/users/0ba95fb65871415ca38c574ddd6396b8 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:51.135920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcb883-67f9-4cd3-853b-10facb3dc867 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:51.136485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcb883-67f9-4cd3-853b-10facb3dc867 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:51.136485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcb883-67f9-4cd3-853b-10facb3dc867 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:51.142349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 5/10] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:08:51.144208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf9f0b-46c4-478e-91d2-4e19239648d0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:51.144675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf9f0b-46c4-478e-91d2-4e19239648d0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:51.144808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf9f0b-46c4-478e-91d2-4e19239648d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:51.163067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-09bf9f0b-46c4-478e-91d2-4e19239648d0 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:51.164250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-09bf9f0b-46c4-478e-91d2-4e19239648d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:51.172792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf2b92-249d-4c6e-b4f7-55127b1f5dd8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:51.173093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf2b92-249d-4c6e-b4f7-55127b1f5dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:51.173426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf2b92-249d-4c6e-b4f7-55127b1f5dd8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:51.175147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 6/11] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:08:51.181200 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4da656-45c8-48ee-a987-08fcf53645b9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:51.181526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4da656-45c8-48ee-a987-08fcf53645b9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:51.181800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4da656-45c8-48ee-a987-08fcf53645b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:51.226492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG passlib.handlers.bcrypt [None req-1e4da656-45c8-48ee-a987-08fcf53645b9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81504) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 19 15:08:51.226910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG passlib.handlers.bcrypt [None req-1e4da656-45c8-48ee-a987-08fcf53645b9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81504) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 19 15:08:51.246148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 6/12] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:08:51.275003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1e4da656-45c8-48ee-a987-08fcf53645b9 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:51.277019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1e4da656-45c8-48ee-a987-08fcf53645b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:51.377781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 7/13] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:08:51.514050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf9f0b-46c4-478e-91d2-4e19239648d0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:51.538056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:51.628668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf9f0b-46c4-478e-91d2-4e19239648d0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:51.651528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4da656-45c8-48ee-a987-08fcf53645b9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:51.662328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c480b-a74a-4703-a5b9-812de19cad07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:51.662969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c480b-a74a-4703-a5b9-812de19cad07 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:51.663253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c480b-a74a-4703-a5b9-812de19cad07 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:51.663533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c480b-a74a-4703-a5b9-812de19cad07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:51.679608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c480b-a74a-4703-a5b9-812de19cad07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-464c480b-a74a-4703-a5b9-812de19cad07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:08:51.688068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:08:51.689809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:08:51.690951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 7/14] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:08:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:08:51.694942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c480b-a74a-4703-a5b9-812de19cad07 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:51.760996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4da656-45c8-48ee-a987-08fcf53645b9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:51.763493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:51.780566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:51.799123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c480b-a74a-4703-a5b9-812de19cad07 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:51.801221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f51066-17d1-4092-92f8-df147e99415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:51.802150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f51066-17d1-4092-92f8-df147e99415c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:51.802439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f51066-17d1-4092-92f8-df147e99415c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:51.802714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f51066-17d1-4092-92f8-df147e99415c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:51.831842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd738c-0dc8-4a13-af9f-80d539488c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:51.832471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd738c-0dc8-4a13-af9f-80d539488c74 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:51.832755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd738c-0dc8-4a13-af9f-80d539488c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:51.833027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd738c-0dc8-4a13-af9f-80d539488c74 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:51.833689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd738c-0dc8-4a13-af9f-80d539488c74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:08:51.833990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd738c-0dc8-4a13-af9f-80d539488c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:51.835406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd738c-0dc8-4a13-af9f-80d539488c74 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:51.847586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 8/15] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:08:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:08:51.853155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd738c-0dc8-4a13-af9f-80d539488c74 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:51.870730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f51066-17d1-4092-92f8-df147e99415c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:51.872451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f51066-17d1-4092-92f8-df147e99415c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:51.953026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd738c-0dc8-4a13-af9f-80d539488c74 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:51.972621 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 8/16] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:08:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:08:51.978388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f51066-17d1-4092-92f8-df147e99415c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:51.981916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:51.990232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042c60-aad3-4552-b285-cad877d5bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:51.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042c60-aad3-4552-b285-cad877d5bc30 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:51.991609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042c60-aad3-4552-b285-cad877d5bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:51.991885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042c60-aad3-4552-b285-cad877d5bc30 None admin] PATH_INFO: `/v3/users/0ba95fb65871415ca38c574ddd6396b8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:51.993056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad042c60-aad3-4552-b285-cad877d5bc30 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba95fb65871415ca38c574ddd6396b8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:51.994594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad042c60-aad3-4552-b285-cad877d5bc30 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:51.996169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 9/17] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:08:51 2022] GET /identity/v3/users/0ba95fb65871415ca38c574ddd6396b8 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:08:52.001868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042c60-aad3-4552-b285-cad877d5bc30 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:52.017050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:52.078956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f51066-17d1-4092-92f8-df147e99415c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:52.104366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042c60-aad3-4552-b285-cad877d5bc30 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:52.112959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d316a1b-b375-4394-89cb-25183cd8e587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:52.113811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d316a1b-b375-4394-89cb-25183cd8e587 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:52.114088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d316a1b-b375-4394-89cb-25183cd8e587 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:52.114360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d316a1b-b375-4394-89cb-25183cd8e587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:52.132973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d316a1b-b375-4394-89cb-25183cd8e587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:52.134466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d316a1b-b375-4394-89cb-25183cd8e587 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:52.136967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe1805-43a0-48a8-990e-7e0ebab0fa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:52.138211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe1805-43a0-48a8-990e-7e0ebab0fa46 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:52.138654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe1805-43a0-48a8-990e-7e0ebab0fa46 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:52.138931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe1805-43a0-48a8-990e-7e0ebab0fa46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:52.140067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe1805-43a0-48a8-990e-7e0ebab0fa46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:52.141870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe1805-43a0-48a8-990e-7e0ebab0fa46 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:52.143353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 10/18] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:08:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:08:52.148959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe1805-43a0-48a8-990e-7e0ebab0fa46 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-1d316a1b-b375-4394-89cb-25183cd8e587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:08:52.154201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:08:52.155513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:08:52.156631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:08:52.156631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:08:52.156631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:08:52.156631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 9/19] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:08:52.160939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d316a1b-b375-4394-89cb-25183cd8e587 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:52.216193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:52.233786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:52.254364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe1805-43a0-48a8-990e-7e0ebab0fa46 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:52.260435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d316a1b-b375-4394-89cb-25183cd8e587 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:52.288234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110b372-8b55-4d7c-a0e1-bd7547a8b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:52.288871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110b372-8b55-4d7c-a0e1-bd7547a8b470 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:52.289158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110b372-8b55-4d7c-a0e1-bd7547a8b470 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:52.289445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110b372-8b55-4d7c-a0e1-bd7547a8b470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:52.290170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110b372-8b55-4d7c-a0e1-bd7547a8b470 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.domain.id=default, user.id=0ba95fb65871415ca38c574ddd6396b8) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:08:52.290480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110b372-8b55-4d7c-a0e1-bd7547a8b470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:52.291953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110b372-8b55-4d7c-a0e1-bd7547a8b470 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:52.294585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663030-77d1-40dc-9f0c-e845942d7430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:52.295185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663030-77d1-40dc-9f0c-e845942d7430 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:52.295466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663030-77d1-40dc-9f0c-e845942d7430 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:52.295740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663030-77d1-40dc-9f0c-e845942d7430 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:52.296404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec663030-77d1-40dc-9f0c-e845942d7430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:08:52.296721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec663030-77d1-40dc-9f0c-e845942d7430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:52.298192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec663030-77d1-40dc-9f0c-e845942d7430 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:52.299148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 11/20] 10.222.0.75 () {56 vars in 1322 bytes} [Fri Aug 19 15:08:52 2022] GET /identity/v3/role_assignments?user.id=0ba95fb65871415ca38c574ddd6396b8&scope.domain.id=default&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:08:52.319702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 10/21] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:08:54.510377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3210c-a415-4531-8010-3aa7c91bb8d7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:54.510893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3210c-a415-4531-8010-3aa7c91bb8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:54.510893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3210c-a415-4531-8010-3aa7c91bb8d7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:54.512555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 12/22] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:08:54.526258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00650f-e508-417b-930c-ee8258800060 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:54.526725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00650f-e508-417b-930c-ee8258800060 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:54.526796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00650f-e508-417b-930c-ee8258800060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:54.561391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bb00650f-e508-417b-930c-ee8258800060 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:54.562601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bb00650f-e508-417b-930c-ee8258800060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:54.649217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 11/23] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:08:54.884471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d3f51-a360-4532-b8ba-17e4a07f21f3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:54.884678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d3f51-a360-4532-b8ba-17e4a07f21f3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:54.885185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d3f51-a360-4532-b8ba-17e4a07f21f3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:54.886695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 13/24] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:08:54.892170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfa1b6-be6f-42a5-a571-1e08bfb768b6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:54.892713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfa1b6-be6f-42a5-a571-1e08bfb768b6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:54.893200 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfa1b6-be6f-42a5-a571-1e08bfb768b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:54.917332 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d3cfa1b6-be6f-42a5-a571-1e08bfb768b6 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:54.919393 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d3cfa1b6-be6f-42a5-a571-1e08bfb768b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:54.934524 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d3c24-327a-48cf-84a6-38d2f0c59f8f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:54.935094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d3c24-327a-48cf-84a6-38d2f0c59f8f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:54.935588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d3c24-327a-48cf-84a6-38d2f0c59f8f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:54.937150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 14/25] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:08:54.942207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4e84e-a916-492d-bef5-440822913739 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:54.942759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4e84e-a916-492d-bef5-440822913739 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:54.943224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4e84e-a916-492d-bef5-440822913739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:54.963268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-23d4e84e-a916-492d-bef5-440822913739 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:54.966486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-23d4e84e-a916-492d-bef5-440822913739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:54.990830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:55.035391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 12/26] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:08:55.097743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 15/27] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:08:55.133855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfa1b6-be6f-42a5-a571-1e08bfb768b6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:55.262686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfa1b6-be6f-42a5-a571-1e08bfb768b6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:55.307134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b755f3-711b-4bbf-b769-fbee1ac5df48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:55.307764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b755f3-711b-4bbf-b769-fbee1ac5df48 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:55.308042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b755f3-711b-4bbf-b769-fbee1ac5df48 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:55.308312 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b755f3-711b-4bbf-b769-fbee1ac5df48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:55.320537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b755f3-711b-4bbf-b769-fbee1ac5df48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-69b755f3-711b-4bbf-b769-fbee1ac5df48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:08:55.327110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:08:55.328011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:08:55.328810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 13/28] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:08:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:08:55.337867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4e84e-a916-492d-bef5-440822913739 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:55.417817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:55.421020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b755f3-711b-4bbf-b769-fbee1ac5df48 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:55.475225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4e84e-a916-492d-bef5-440822913739 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:55.480376 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:55.532697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532b4c0-4b37-4650-ab10-592bc87499ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:55.533992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532b4c0-4b37-4650-ab10-592bc87499ab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:55.534465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532b4c0-4b37-4650-ab10-592bc87499ab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:55.534928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532b4c0-4b37-4650-ab10-592bc87499ab None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:55.535770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532b4c0-4b37-4650-ab10-592bc87499ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:08:55.536249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532b4c0-4b37-4650-ab10-592bc87499ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:55.537659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532b4c0-4b37-4650-ab10-592bc87499ab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:55.547206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 16/29] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:08:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:55.548643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d36ba-3085-4366-8ce5-99fe51c5ddb8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:55.549081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d36ba-3085-4366-8ce5-99fe51c5ddb8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:55.549488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d36ba-3085-4366-8ce5-99fe51c5ddb8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:55.550801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 17/30] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:08:55.551676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d36ba-3085-4366-8ce5-99fe51c5ddb8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:55.573928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b755f3-711b-4bbf-b769-fbee1ac5df48 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:55.618813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d2288-3dfd-4082-959f-8a50e223cf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:55.619625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d2288-3dfd-4082-959f-8a50e223cf08 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:55.620314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d2288-3dfd-4082-959f-8a50e223cf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:55.620623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d2288-3dfd-4082-959f-8a50e223cf08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:55.639285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d2288-3dfd-4082-959f-8a50e223cf08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:55.641432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d2288-3dfd-4082-959f-8a50e223cf08 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-fe9d2288-3dfd-4082-959f-8a50e223cf08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:08:55.664507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:08:55.665436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:08:55.666311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:08:55.666311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:08:55.666311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:08:55.666311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 14/31] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:08:55.666952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d2288-3dfd-4082-959f-8a50e223cf08 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:55.687236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d36ba-3085-4366-8ce5-99fe51c5ddb8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:55.739874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d253d-fa43-477c-967e-3d9ffe84af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:55.740563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d253d-fa43-477c-967e-3d9ffe84af1d None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:55.740806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d253d-fa43-477c-967e-3d9ffe84af1d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:55.741124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d253d-fa43-477c-967e-3d9ffe84af1d None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:55.741790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d253d-fa43-477c-967e-3d9ffe84af1d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:55.743234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d253d-fa43-477c-967e-3d9ffe84af1d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:55.753978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:55.789935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:55.803732 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d2288-3dfd-4082-959f-8a50e223cf08 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:55.841869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a1772-2be3-4d67-bdf8-57b3ab49bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:55.842896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1772-2be3-4d67-bdf8-57b3ab49bba5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:55.843393 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1772-2be3-4d67-bdf8-57b3ab49bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:55.843872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1772-2be3-4d67-bdf8-57b3ab49bba5 None admin] PATH_INFO: `/v3/users/0ba95fb65871415ca38c574ddd6396b8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:55.845630 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a1772-2be3-4d67-bdf8-57b3ab49bba5 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba95fb65871415ca38c574ddd6396b8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:55.847322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a1772-2be3-4d67-bdf8-57b3ab49bba5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:55.849135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 15/32] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:08:55 2022] GET /identity/v3/users/0ba95fb65871415ca38c574ddd6396b8 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:55.850588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ebb48-21bd-4b1f-b39e-a181a7e265da None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:55.851064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ebb48-21bd-4b1f-b39e-a181a7e265da None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:55.851529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ebb48-21bd-4b1f-b39e-a181a7e265da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:55.872707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b70ebb48-21bd-4b1f-b39e-a181a7e265da None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:55.874309 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b70ebb48-21bd-4b1f-b39e-a181a7e265da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:55.918037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 18/33] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Aug 19 15:08:55 2022] POST /identity/v3/roles => generated 219 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:08:55.923980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d253d-fa43-477c-967e-3d9ffe84af1d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:55.983917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 16/34] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:08:55.986022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ebb48-21bd-4b1f-b39e-a181a7e265da None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:56.117368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d253d-fa43-477c-967e-3d9ffe84af1d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:56.146145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:56.166219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ebb48-21bd-4b1f-b39e-a181a7e265da None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:56.171516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacd07d-490d-4543-85ab-03772c69267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:56.171516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacd07d-490d-4543-85ab-03772c69267d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:56.171516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacd07d-490d-4543-85ab-03772c69267d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:56.171829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacd07d-490d-4543-85ab-03772c69267d None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:56.172289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacd07d-490d-4543-85ab-03772c69267d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:08:56.172646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacd07d-490d-4543-85ab-03772c69267d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:56.174544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacd07d-490d-4543-85ab-03772c69267d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:56.196013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 19/35] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:56.203126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacd07d-490d-4543-85ab-03772c69267d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:56.226777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762ca2a-81f2-4b78-96ba-54917330e742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:56.227764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762ca2a-81f2-4b78-96ba-54917330e742 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:56.229039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762ca2a-81f2-4b78-96ba-54917330e742 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:56.229541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762ca2a-81f2-4b78-96ba-54917330e742 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:56.233056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762ca2a-81f2-4b78-96ba-54917330e742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:56.235639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762ca2a-81f2-4b78-96ba-54917330e742 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:56.240201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 17/36] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:08:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:56.244383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762ca2a-81f2-4b78-96ba-54917330e742 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:56.315485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:56.373417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacd07d-490d-4543-85ab-03772c69267d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:56.382203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762ca2a-81f2-4b78-96ba-54917330e742 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:56.413092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc14fc0-e26c-4f3c-a8c0-b970086afbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:56.413766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14fc0-e26c-4f3c-a8c0-b970086afbd1 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:56.414003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14fc0-e26c-4f3c-a8c0-b970086afbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:56.414280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc14fc0-e26c-4f3c-a8c0-b970086afbd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:56.414924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc14fc0-e26c-4f3c-a8c0-b970086afbd1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:56.416424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc14fc0-e26c-4f3c-a8c0-b970086afbd1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:56.419257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14193259-fa38-4e44-b8c0-32b0b116065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:56.419921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193259-fa38-4e44-b8c0-32b0b116065b None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:56.420206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193259-fa38-4e44-b8c0-32b0b116065b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:56.420519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193259-fa38-4e44-b8c0-32b0b116065b None admin] PATH_INFO: `/v3/domains/default/users/0ba95fb65871415ca38c574ddd6396b8/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:56.422477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14193259-fa38-4e44-b8c0-32b0b116065b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0ba95fb65871415ca38c574ddd6396b8, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:56.423927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14193259-fa38-4e44-b8c0-32b0b116065b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:56.450575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 20/37] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:08:56 2022] POST /identity/v3/projects => generated 301 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:08:56.451958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808bfd-019d-437c-a16e-b4f34c281951 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:56.452256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808bfd-019d-437c-a16e-b4f34c281951 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:56.452570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99808bfd-019d-437c-a16e-b4f34c281951 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:56.458111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 21/38] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:08:56.458111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808bfd-019d-437c-a16e-b4f34c281951 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:56.546281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 18/39] 10.222.0.75 () {58 vars in 1361 bytes} [Fri Aug 19 15:08:56 2022] PUT /identity/v3/domains/default/users/0ba95fb65871415ca38c574ddd6396b8/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:08:56.548127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63beb974-31cf-47f7-bda0-2504c3d87100 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:56.548449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63beb974-31cf-47f7-bda0-2504c3d87100 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:56.548734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63beb974-31cf-47f7-bda0-2504c3d87100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:56.569223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-63beb974-31cf-47f7-bda0-2504c3d87100 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:56.570869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-63beb974-31cf-47f7-bda0-2504c3d87100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:56.583369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:56.605852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99808bfd-019d-437c-a16e-b4f34c281951 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:56.618362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:56.662785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8644e-37de-48c7-ab3f-1173b62a8b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:56.663324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8644e-37de-48c7-ab3f-1173b62a8b8d None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:56.663612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8644e-37de-48c7-ab3f-1173b62a8b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:56.663890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8644e-37de-48c7-ab3f-1173b62a8b8d None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:56.665218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8644e-37de-48c7-ab3f-1173b62a8b8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:56.666751 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8644e-37de-48c7-ab3f-1173b62a8b8d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:56.683332 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 19/40] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:08:56.685129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a7c12-d574-4e7b-81df-780ac9888ed2 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:56.685433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a7c12-d574-4e7b-81df-780ac9888ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:56.685721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a7c12-d574-4e7b-81df-780ac9888ed2 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:56.689988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 20/41] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:08:56.694704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 22/42] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Aug 19 15:08:56 2022] POST /identity/v3/roles => generated 225 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:08:56.698837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01e0a3-f1d2-4869-86d8-79ff95bad04b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:56.698837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01e0a3-f1d2-4869-86d8-79ff95bad04b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:56.698837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01e0a3-f1d2-4869-86d8-79ff95bad04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:56.721380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-eb01e0a3-f1d2-4869-86d8-79ff95bad04b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:56.723327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-eb01e0a3-f1d2-4869-86d8-79ff95bad04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:56.811191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 21/43] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:08:56.975333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8644e-37de-48c7-ab3f-1173b62a8b8d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:57.081656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8644e-37de-48c7-ab3f-1173b62a8b8d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:57.084139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01e0a3-f1d2-4869-86d8-79ff95bad04b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:57.122006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49474d6-bfb1-4867-b965-0843658ed8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:57.122839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49474d6-bfb1-4867-b965-0843658ed8b1 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:57.123125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49474d6-bfb1-4867-b965-0843658ed8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:57.123405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49474d6-bfb1-4867-b965-0843658ed8b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:57.124077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49474d6-bfb1-4867-b965-0843658ed8b1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:57.126022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49474d6-bfb1-4867-b965-0843658ed8b1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:57.143228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 23/44] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Aug 19 15:08:56 2022] POST /identity/v3/roles => generated 223 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:08:57.175145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:57.204925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01e0a3-f1d2-4869-86d8-79ff95bad04b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:57.242432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21243d-d394-46cd-bccf-1c327ab48c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:57.243001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21243d-d394-46cd-bccf-1c327ab48c89 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:57.243290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21243d-d394-46cd-bccf-1c327ab48c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:57.243571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21243d-d394-46cd-bccf-1c327ab48c89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:57.244788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21243d-d394-46cd-bccf-1c327ab48c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:57.247009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21243d-d394-46cd-bccf-1c327ab48c89 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:57.248965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 22/45] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:08:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:08:57.254621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49474d6-bfb1-4867-b965-0843658ed8b1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:57.287058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:57.366035 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49474d6-bfb1-4867-b965-0843658ed8b1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:57.398800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362043a-ea9a-4025-98a2-40a55c7daca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:57.399411 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362043a-ea9a-4025-98a2-40a55c7daca3 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:57.399699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362043a-ea9a-4025-98a2-40a55c7daca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:57.399977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362043a-ea9a-4025-98a2-40a55c7daca3 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:57.400917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362043a-ea9a-4025-98a2-40a55c7daca3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:57.402402 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362043a-ea9a-4025-98a2-40a55c7daca3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:57.428976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 24/46] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:08:57 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:08:58.570015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b9364-0af1-40d7-8dd0-2b02072e4c16 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:58.570535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b9364-0af1-40d7-8dd0-2b02072e4c16 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:58.570535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b9364-0af1-40d7-8dd0-2b02072e4c16 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:58.572146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 23/47] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:08:58.578220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3ae82-d1da-4795-af2a-33c27eb3386e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:58.578624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3ae82-d1da-4795-af2a-33c27eb3386e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:58.578785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3ae82-d1da-4795-af2a-33c27eb3386e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:58.598062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-51e3ae82-d1da-4795-af2a-33c27eb3386e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:58.599589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-51e3ae82-d1da-4795-af2a-33c27eb3386e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:58.772291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 25/48] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:08:59.044734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b9364-0af1-40d7-8dd0-2b02072e4c16 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:59.146834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b9364-0af1-40d7-8dd0-2b02072e4c16 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:59.180822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f48389-6e98-43eb-b792-08deff38b741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:59.181443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f48389-6e98-43eb-b792-08deff38b741 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:59.181724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f48389-6e98-43eb-b792-08deff38b741 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:59.181997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f48389-6e98-43eb-b792-08deff38b741 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:59.192771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f48389-6e98-43eb-b792-08deff38b741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e0f48389-6e98-43eb-b792-08deff38b741 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:08:59.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:08:59.198993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:08:59.200911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 24/49] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:08:59.205096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3ae82-d1da-4795-af2a-33c27eb3386e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:59.217080 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:59.305588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3ae82-d1da-4795-af2a-33c27eb3386e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:59.323089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e866b1a-6ddd-4cf0-80cd-15233473e5b4 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:59.323379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e866b1a-6ddd-4cf0-80cd-15233473e5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:59.323656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e866b1a-6ddd-4cf0-80cd-15233473e5b4 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:59.325759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 25/50] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:08:59.331408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ba6b2-1453-4cd1-840b-89fc4b5f6aa0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:59.331693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ba6b2-1453-4cd1-840b-89fc4b5f6aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:59.331970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ba6b2-1453-4cd1-840b-89fc4b5f6aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:59.338085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:59.341915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14ba37-282b-46e0-8997-a42c428034c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:59.342578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14ba37-282b-46e0-8997-a42c428034c4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:59.342880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14ba37-282b-46e0-8997-a42c428034c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:59.343157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14ba37-282b-46e0-8997-a42c428034c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:59.343838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14ba37-282b-46e0-8997-a42c428034c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:08:59.344141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14ba37-282b-46e0-8997-a42c428034c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:59.345565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14ba37-282b-46e0-8997-a42c428034c4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:59.350503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-340ba6b2-1453-4cd1-840b-89fc4b5f6aa0 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:08:59.351707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-340ba6b2-1453-4cd1-840b-89fc4b5f6aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:08:59.354615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 26/51] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:59.360087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14ba37-282b-46e0-8997-a42c428034c4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:59.420012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 26/52] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:08:59.430297 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:59.461768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14ba37-282b-46e0-8997-a42c428034c4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:59.494290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5bab2-97c1-45bf-9234-882df4dfdecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:59.494917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5bab2-97c1-45bf-9234-882df4dfdecb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:59.495198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5bab2-97c1-45bf-9234-882df4dfdecb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:59.495470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5bab2-97c1-45bf-9234-882df4dfdecb None admin] PATH_INFO: `/v3/users/0ba95fb65871415ca38c574ddd6396b8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:59.497306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5bab2-97c1-45bf-9234-882df4dfdecb None admin] RBAC: Authorizing `identity:get_user(user_id=0ba95fb65871415ca38c574ddd6396b8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:59.498887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5bab2-97c1-45bf-9234-882df4dfdecb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:59.500477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 27/53] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:08:59 2022] GET /identity/v3/users/0ba95fb65871415ca38c574ddd6396b8 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:08:59.505973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ba6b2-1453-4cd1-840b-89fc4b5f6aa0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:59.602445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ba6b2-1453-4cd1-840b-89fc4b5f6aa0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:59.631596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:59.637200 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eee285-818c-436e-8446-df49ab468c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:59.638008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eee285-818c-436e-8446-df49ab468c34 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:59.638289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eee285-818c-436e-8446-df49ab468c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:59.638562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eee285-818c-436e-8446-df49ab468c34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:59.639659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eee285-818c-436e-8446-df49ab468c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:59.641157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eee285-818c-436e-8446-df49ab468c34 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:59.642870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 27/54] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:08:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:08:59.648427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5bab2-97c1-45bf-9234-882df4dfdecb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:59.699035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eee285-818c-436e-8446-df49ab468c34 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:59.754395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5bab2-97c1-45bf-9234-882df4dfdecb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:59.789125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc733c5d-0242-42a1-afb3-a347c8b117a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:59.789729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc733c5d-0242-42a1-afb3-a347c8b117a6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:59.790008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc733c5d-0242-42a1-afb3-a347c8b117a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:59.790283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc733c5d-0242-42a1-afb3-a347c8b117a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:59.791005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc733c5d-0242-42a1-afb3-a347c8b117a6 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.domain.id=default, user.id=0ba95fb65871415ca38c574ddd6396b8) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:08:59.791311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc733c5d-0242-42a1-afb3-a347c8b117a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:59.792800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc733c5d-0242-42a1-afb3-a347c8b117a6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:59.796310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eee285-818c-436e-8446-df49ab468c34 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:59.799486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 28/55] 10.222.0.75 () {56 vars in 1322 bytes} [Fri Aug 19 15:08:59 2022] GET /identity/v3/role_assignments?user.id=0ba95fb65871415ca38c574ddd6396b8&scope.domain.id=default&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:59.833494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa4864-a36f-434b-8145-2322a089f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:59.834228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa4864-a36f-434b-8145-2322a089f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:59.834508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa4864-a36f-434b-8145-2322a089f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:59.834780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa4864-a36f-434b-8145-2322a089f6f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:59.835971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa4864-a36f-434b-8145-2322a089f6f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:59.837493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa4864-a36f-434b-8145-2322a089f6f0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:08:59.839097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 28/56] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:08:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:08:59.845156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc733c5d-0242-42a1-afb3-a347c8b117a6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:08:59.849236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:08:59.959461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc733c5d-0242-42a1-afb3-a347c8b117a6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:08:59.995652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2200d52-9155-486d-b90b-b6b0b63e7110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:08:59.996307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200d52-9155-486d-b90b-b6b0b63e7110 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:08:59.996623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200d52-9155-486d-b90b-b6b0b63e7110 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:08:59.996952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200d52-9155-486d-b90b-b6b0b63e7110 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:08:59.997611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2200d52-9155-486d-b90b-b6b0b63e7110 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:08:59.999076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2200d52-9155-486d-b90b-b6b0b63e7110 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:00.025171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 29/57] 10.222.0.75 () {60 vars in 1140 bytes} [Fri Aug 19 15:08:59 2022] POST /identity/v3/projects => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:09:01.928275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bd0bd-d874-496c-bbd0-5ffbff8e26bf None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:01.928805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bd0bd-d874-496c-bbd0-5ffbff8e26bf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:01.928805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bd0bd-d874-496c-bbd0-5ffbff8e26bf None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:01.930125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 29/58] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:01.935956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff823df2-cd5c-4903-8f74-81f1005f8e95 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:01.936361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff823df2-cd5c-4903-8f74-81f1005f8e95 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:01.936539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff823df2-cd5c-4903-8f74-81f1005f8e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:01.954140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ff823df2-cd5c-4903-8f74-81f1005f8e95 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:01.955321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ff823df2-cd5c-4903-8f74-81f1005f8e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:02.008467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:02.026095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 30/59] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:02.314073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bd0bd-d874-496c-bbd0-5ffbff8e26bf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:02.380033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:02.414305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bd0bd-d874-496c-bbd0-5ffbff8e26bf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:02.448497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c7a94-37b5-4ebb-8461-4405b23a1d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:02.449108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c7a94-37b5-4ebb-8461-4405b23a1d08 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:02.449938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c7a94-37b5-4ebb-8461-4405b23a1d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:02.450703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c7a94-37b5-4ebb-8461-4405b23a1d08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:02.452029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c7a94-37b5-4ebb-8461-4405b23a1d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:02.453631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c7a94-37b5-4ebb-8461-4405b23a1d08 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:02.455183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 30/60] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:02.460710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff823df2-cd5c-4903-8f74-81f1005f8e95 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:02.562539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff823df2-cd5c-4903-8f74-81f1005f8e95 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:02.595912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb34f67-6a56-45b3-8712-948382d39fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:02.596472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34f67-6a56-45b3-8712-948382d39fff None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:02.596754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34f67-6a56-45b3-8712-948382d39fff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:02.597027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34f67-6a56-45b3-8712-948382d39fff None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:02.597663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb34f67-6a56-45b3-8712-948382d39fff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:02.599160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb34f67-6a56-45b3-8712-948382d39fff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:02.627085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 31/61] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:09:02 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:09:05.666907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bd744-10c1-4a78-b730-ab130ada87c7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:05.667434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bd744-10c1-4a78-b730-ab130ada87c7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:05.667434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bd744-10c1-4a78-b730-ab130ada87c7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:05.670634 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 31/62] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:05.682204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9b2b-f2d6-4af6-8a45-2cec68fe76cd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:05.682693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9b2b-f2d6-4af6-8a45-2cec68fe76cd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:05.682781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9b2b-f2d6-4af6-8a45-2cec68fe76cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:05.709463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-79bd9b2b-f2d6-4af6-8a45-2cec68fe76cd None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:05.710685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-79bd9b2b-f2d6-4af6-8a45-2cec68fe76cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:05.765661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:05.810763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 32/63] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:05.881046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b548a7b-f7e9-42ea-8ffc-857063d7467c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:05.881360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b548a7b-f7e9-42ea-8ffc-857063d7467c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:05.881611 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b548a7b-f7e9-42ea-8ffc-857063d7467c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:05.883123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 32/64] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:05.889223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cf8fa-0420-446a-a09f-a82f4729fb35 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:05.889755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cf8fa-0420-446a-a09f-a82f4729fb35 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:05.890225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cf8fa-0420-446a-a09f-a82f4729fb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:05.909262 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dc9cf8fa-0420-446a-a09f-a82f4729fb35 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:05.910771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc9cf8fa-0420-446a-a09f-a82f4729fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:05.992362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 33/65] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:06.011629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43264fb-cfe3-4864-85f1-4797a100f8c1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.012587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43264fb-cfe3-4864-85f1-4797a100f8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.012870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43264fb-cfe3-4864-85f1-4797a100f8c1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.014233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 33/66] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:06.019433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9c301-f010-4595-8277-0d0485ec4528 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.019745 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9c301-f010-4595-8277-0d0485ec4528 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.020025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9c301-f010-4595-8277-0d0485ec4528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.046382 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c8d9c301-f010-4595-8277-0d0485ec4528 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:06.047608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c8d9c301-f010-4595-8277-0d0485ec4528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:06.074035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43262dd7-b780-4314-be62-31a7e6afed29 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.074980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43262dd7-b780-4314-be62-31a7e6afed29 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.075448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43262dd7-b780-4314-be62-31a7e6afed29 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.076839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 34/67] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:09:06.081682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8875b05-ff5f-458e-ac95-8d02666e8c1d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.081967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8875b05-ff5f-458e-ac95-8d02666e8c1d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.082308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8875b05-ff5f-458e-ac95-8d02666e8c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.106819 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e8875b05-ff5f-458e-ac95-8d02666e8c1d None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:06.108557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e8875b05-ff5f-458e-ac95-8d02666e8c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:06.171365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 34/68] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:06.193347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9c301-f010-4595-8277-0d0485ec4528 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:06.248804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 35/69] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:06.249077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:06.294183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8875b05-ff5f-458e-ac95-8d02666e8c1d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:06.339122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d9c301-f010-4595-8277-0d0485ec4528 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:06.377255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:06.379653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b8e85-8564-45a7-8bf9-3b1de4db2852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:06.380598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b8e85-8564-45a7-8bf9-3b1de4db2852 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.381024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b8e85-8564-45a7-8bf9-3b1de4db2852 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.381304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b8e85-8564-45a7-8bf9-3b1de4db2852 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.393701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b8e85-8564-45a7-8bf9-3b1de4db2852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-dc6b8e85-8564-45a7-8bf9-3b1de4db2852 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:06.398953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:06.399974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:06.401264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 35/70] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:06.407676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0b84-ee95-49d8-b2f0-3be976f3a0a8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.407676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0b84-ee95-49d8-b2f0-3be976f3a0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.407676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c0b84-ee95-49d8-b2f0-3be976f3a0a8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.407676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 36/71] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:06.408362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0b84-ee95-49d8-b2f0-3be976f3a0a8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:06.431129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8875b05-ff5f-458e-ac95-8d02666e8c1d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:06.474690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba195a8-1d0b-46cd-b8af-bf3da4bb2d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:06.475356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba195a8-1d0b-46cd-b8af-bf3da4bb2d65 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.475688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba195a8-1d0b-46cd-b8af-bf3da4bb2d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.475972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba195a8-1d0b-46cd-b8af-bf3da4bb2d65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.487573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba195a8-1d0b-46cd-b8af-bf3da4bb2d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6ba195a8-1d0b-46cd-b8af-bf3da4bb2d65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:06.492542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:06.493381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:06.494160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:06.499059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 36/72] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:09:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:06.500733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02639639-f9e5-4826-aab0-1655687622b7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.501078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02639639-f9e5-4826-aab0-1655687622b7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.501364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02639639-f9e5-4826-aab0-1655687622b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.510052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:06.521499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-02639639-f9e5-4826-aab0-1655687622b7 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:06.522695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-02639639-f9e5-4826-aab0-1655687622b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:06.526130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c0b84-ee95-49d8-b2f0-3be976f3a0a8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:06.561359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ed213-1052-48eb-80e3-bff4f4a649d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:06.561975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ed213-1052-48eb-80e3-bff4f4a649d8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.562253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ed213-1052-48eb-80e3-bff4f4a649d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.562530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ed213-1052-48eb-80e3-bff4f4a649d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.563200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ed213-1052-48eb-80e3-bff4f4a649d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:06.563500 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ed213-1052-48eb-80e3-bff4f4a649d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:06.564952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ed213-1052-48eb-80e3-bff4f4a649d8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:06.573120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 37/73] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:06.573913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ed213-1052-48eb-80e3-bff4f4a649d8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:06.598188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 37/74] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:06.599048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02639639-f9e5-4826-aab0-1655687622b7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:06.658757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:06.697629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ed213-1052-48eb-80e3-bff4f4a649d8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:06.713440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02639639-f9e5-4826-aab0-1655687622b7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:06.731836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa85887-021c-4772-a184-74d15fbe5740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:06.732494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa85887-021c-4772-a184-74d15fbe5740 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.732777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa85887-021c-4772-a184-74d15fbe5740 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.733050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa85887-021c-4772-a184-74d15fbe5740 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.742987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa85887-021c-4772-a184-74d15fbe5740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:06.748011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f336-8d6f-4fbc-8e6e-0bccd2af1b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:06.748653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f336-8d6f-4fbc-8e6e-0bccd2af1b60 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4fa85887-021c-4772-a184-74d15fbe5740 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:06.748834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:06.749692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:06.750471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:06.751104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f336-8d6f-4fbc-8e6e-0bccd2af1b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.751104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f336-8d6f-4fbc-8e6e-0bccd2af1b60 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.751104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42f336-8d6f-4fbc-8e6e-0bccd2af1b60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:06.751267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 38/75] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:06.751267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa85887-021c-4772-a184-74d15fbe5740 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:06.752846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42f336-8d6f-4fbc-8e6e-0bccd2af1b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:06.754263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42f336-8d6f-4fbc-8e6e-0bccd2af1b60 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:06.768245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 38/76] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:06.769097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f336-8d6f-4fbc-8e6e-0bccd2af1b60 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:06.775615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:06.872733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa85887-021c-4772-a184-74d15fbe5740 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:06.910227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbcbc2-96de-43dd-89c0-473db2701bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:06.910953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbcbc2-96de-43dd-89c0-473db2701bbc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:06.911232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbcbc2-96de-43dd-89c0-473db2701bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:06.911508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbcbc2-96de-43dd-89c0-473db2701bbc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:06.923039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbcbc2-96de-43dd-89c0-473db2701bbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-94dbcbc2-96de-43dd-89c0-473db2701bbc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:06.928158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:06.929027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:06.929977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 39/77] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:06.930770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbcbc2-96de-43dd-89c0-473db2701bbc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:06.980287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f336-8d6f-4fbc-8e6e-0bccd2af1b60 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:06.997616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.005658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.015192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da36e54-fd5d-407f-bf51-ca601c6f3daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.015828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da36e54-fd5d-407f-bf51-ca601c6f3daf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.016113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da36e54-fd5d-407f-bf51-ca601c6f3daf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.016424 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da36e54-fd5d-407f-bf51-ca601c6f3daf None admin] PATH_INFO: `/v3/users/0c47d47164cb4842b0a2341a28ea82f3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.032862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da36e54-fd5d-407f-bf51-ca601c6f3daf None admin] RBAC: Authorizing `identity:get_user(user_id=0c47d47164cb4842b0a2341a28ea82f3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.033015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbcbc2-96de-43dd-89c0-473db2701bbc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.034351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da36e54-fd5d-407f-bf51-ca601c6f3daf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.035999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 39/78] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:06 2022] GET /identity/v3/users/0c47d47164cb4842b0a2341a28ea82f3 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:07.040524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da36e54-fd5d-407f-bf51-ca601c6f3daf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.066681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e6da8-a0d5-4d72-88ca-7c9ebc37ee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.067287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e6da8-a0d5-4d72-88ca-7c9ebc37ee91 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.067561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e6da8-a0d5-4d72-88ca-7c9ebc37ee91 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.067835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e6da8-a0d5-4d72-88ca-7c9ebc37ee91 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.068527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e6da8-a0d5-4d72-88ca-7c9ebc37ee91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:07.069072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e6da8-a0d5-4d72-88ca-7c9ebc37ee91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.070228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e6da8-a0d5-4d72-88ca-7c9ebc37ee91 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.078630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 40/79] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:07.082968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e6da8-a0d5-4d72-88ca-7c9ebc37ee91 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.141177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da36e54-fd5d-407f-bf51-ca601c6f3daf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.175233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d63c62-a9d3-40df-8b99-813424d96df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.175878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d63c62-a9d3-40df-8b99-813424d96df2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.176173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d63c62-a9d3-40df-8b99-813424d96df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.176544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d63c62-a9d3-40df-8b99-813424d96df2 None admin] PATH_INFO: `/v3/users/0ba95fb65871415ca38c574ddd6396b8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.177688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d63c62-a9d3-40df-8b99-813424d96df2 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba95fb65871415ca38c574ddd6396b8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.179148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d63c62-a9d3-40df-8b99-813424d96df2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.180770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 40/80] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:07 2022] GET /identity/v3/users/0ba95fb65871415ca38c574ddd6396b8 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:07.182114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e6da8-a0d5-4d72-88ca-7c9ebc37ee91 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.182605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d63c62-a9d3-40df-8b99-813424d96df2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.216665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e020196-9e7a-426e-b425-00448daae489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.217309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e020196-9e7a-426e-b425-00448daae489 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.217596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e020196-9e7a-426e-b425-00448daae489 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.217878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e020196-9e7a-426e-b425-00448daae489 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.218998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e020196-9e7a-426e-b425-00448daae489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.220511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e020196-9e7a-426e-b425-00448daae489 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.222099 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 41/81] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:07.222865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e020196-9e7a-426e-b425-00448daae489 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.226374 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.233074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.287398 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d63c62-a9d3-40df-8b99-813424d96df2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.321775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec41b5-a07e-4149-ad0e-bdc6a5c37b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.322413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec41b5-a07e-4149-ad0e-bdc6a5c37b11 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.322700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec41b5-a07e-4149-ad0e-bdc6a5c37b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.322802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e020196-9e7a-426e-b425-00448daae489 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.322980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec41b5-a07e-4149-ad0e-bdc6a5c37b11 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.323649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec41b5-a07e-4149-ad0e-bdc6a5c37b11 None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:07.323955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec41b5-a07e-4149-ad0e-bdc6a5c37b11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.325384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec41b5-a07e-4149-ad0e-bdc6a5c37b11 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.333676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 41/82] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:07.334465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec41b5-a07e-4149-ad0e-bdc6a5c37b11 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.361492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1f5c1-93a6-4041-989e-b4360c83277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.362097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1f5c1-93a6-4041-989e-b4360c83277c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.362376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1f5c1-93a6-4041-989e-b4360c83277c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.362650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1f5c1-93a6-4041-989e-b4360c83277c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.430829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1f5c1-93a6-4041-989e-b4360c83277c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.432312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1f5c1-93a6-4041-989e-b4360c83277c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.434909 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.436477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec41b5-a07e-4149-ad0e-bdc6a5c37b11 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.443427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.472353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9740c8-19a5-4114-b725-9301eaf10cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.473011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9740c8-19a5-4114-b725-9301eaf10cfd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.473292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9740c8-19a5-4114-b725-9301eaf10cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.473564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9740c8-19a5-4114-b725-9301eaf10cfd None admin] PATH_INFO: `/v3/users/0c47d47164cb4842b0a2341a28ea82f3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.474710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9740c8-19a5-4114-b725-9301eaf10cfd None admin] RBAC: Authorizing `identity:get_user(user_id=0c47d47164cb4842b0a2341a28ea82f3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.476173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9740c8-19a5-4114-b725-9301eaf10cfd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.477809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 42/83] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:07 2022] GET /identity/v3/users/0c47d47164cb4842b0a2341a28ea82f3 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:07.483845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9740c8-19a5-4114-b725-9301eaf10cfd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.534284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 42/84] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:07.535115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1f5c1-93a6-4041-989e-b4360c83277c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.582144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9740c8-19a5-4114-b725-9301eaf10cfd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.619298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d521-1d63-441d-9471-617cdff58b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.620125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d521-1d63-441d-9471-617cdff58b80 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.620434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d521-1d63-441d-9471-617cdff58b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.620720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d521-1d63-441d-9471-617cdff58b80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.633499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1f5c1-93a6-4041-989e-b4360c83277c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.635750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.666063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.673599 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf0976-fe4d-43cb-ad09-9f6cdcd9444c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.674240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf0976-fe4d-43cb-ad09-9f6cdcd9444c None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.674507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf0976-fe4d-43cb-ad09-9f6cdcd9444c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.674793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf0976-fe4d-43cb-ad09-9f6cdcd9444c None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.675457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf0976-fe4d-43cb-ad09-9f6cdcd9444c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.676959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf0976-fe4d-43cb-ad09-9f6cdcd9444c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.695744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1d521-1d63-441d-9471-617cdff58b80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.697267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1d521-1d63-441d-9471-617cdff58b80 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.697768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 43/85] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:09:07 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:09:07.698589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf0976-fe4d-43cb-ad09-9f6cdcd9444c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.817403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 43/86] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:07.818279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d521-1d63-441d-9471-617cdff58b80 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.827467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf0976-fe4d-43cb-ad09-9f6cdcd9444c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.870681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c838679-6b5e-47bf-bfb6-5fd9381ef7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.870919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c838679-6b5e-47bf-bfb6-5fd9381ef7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:07.871232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c838679-6b5e-47bf-bfb6-5fd9381ef7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:07.871845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c838679-6b5e-47bf-bfb6-5fd9381ef7a4 None admin] PATH_INFO: `/v3/users/0c47d47164cb4842b0a2341a28ea82f3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:07.873056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c838679-6b5e-47bf-bfb6-5fd9381ef7a4 None admin] RBAC: Authorizing `identity:get_user(user_id=0c47d47164cb4842b0a2341a28ea82f3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:07.874553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c838679-6b5e-47bf-bfb6-5fd9381ef7a4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:07.876180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 44/87] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:07 2022] GET /identity/v3/users/0c47d47164cb4842b0a2341a28ea82f3 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:07.879113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c838679-6b5e-47bf-bfb6-5fd9381ef7a4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:07.912283 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.917735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:07.945667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d521-1d63-441d-9471-617cdff58b80 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:07.999009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e662ba6-e748-4608-8b4f-f8379ec2d761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:07.999940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e662ba6-e748-4608-8b4f-f8379ec2d761 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:08.000238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e662ba6-e748-4608-8b4f-f8379ec2d761 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:08.000561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e662ba6-e748-4608-8b4f-f8379ec2d761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:08.018989 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c838679-6b5e-47bf-bfb6-5fd9381ef7a4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:08.059126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04295-16da-44f4-abe5-c335f90de9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:08.059775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04295-16da-44f4-abe5-c335f90de9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:08.060060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04295-16da-44f4-abe5-c335f90de9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:08.060337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04295-16da-44f4-abe5-c335f90de9f7 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:08.061738 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a04295-16da-44f4-abe5-c335f90de9f7 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:08.063293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a04295-16da-44f4-abe5-c335f90de9f7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:08.065670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 45/88] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:07 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:08.066460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04295-16da-44f4-abe5-c335f90de9f7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:08.085085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e662ba6-e748-4608-8b4f-f8379ec2d761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:08.086732 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e662ba6-e748-4608-8b4f-f8379ec2d761 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:08.180725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:08.203723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:08.209867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 44/89] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:08.210545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e662ba6-e748-4608-8b4f-f8379ec2d761 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:08.212434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04295-16da-44f4-abe5-c335f90de9f7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:08.258754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a24897-5fd8-4161-b49a-09d835b1a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:08.259454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a24897-5fd8-4161-b49a-09d835b1a952 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:08.259981 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a24897-5fd8-4161-b49a-09d835b1a952 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:08.260194 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a24897-5fd8-4161-b49a-09d835b1a952 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:08.261179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a24897-5fd8-4161-b49a-09d835b1a952 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:08.263235 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a24897-5fd8-4161-b49a-09d835b1a952 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:08.265245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 46/90] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:08 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:08.266226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a24897-5fd8-4161-b49a-09d835b1a952 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:08.344381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e662ba6-e748-4608-8b4f-f8379ec2d761 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:08.377260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a24897-5fd8-4161-b49a-09d835b1a952 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:08.383294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fde1eb-d7e5-48bc-9f2f-22e8413a8490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:08.384095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde1eb-d7e5-48bc-9f2f-22e8413a8490 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:08.384652 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde1eb-d7e5-48bc-9f2f-22e8413a8490 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:08.385101 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde1eb-d7e5-48bc-9f2f-22e8413a8490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:08.419610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fcde3-dad1-4921-af20-cce10f74f063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:08.420107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fcde3-dad1-4921-af20-cce10f74f063 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:08.420416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fcde3-dad1-4921-af20-cce10f74f063 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:08.420717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fcde3-dad1-4921-af20-cce10f74f063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:08.421502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fcde3-dad1-4921-af20-cce10f74f063 None admin] RBAC: Adding query filter params (role.id=a7f0f98b4677401690c354299b049892, scope.project.id=c61f47242a0d40f59b342520453f190d, user.id=0c47d47164cb4842b0a2341a28ea82f3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:08.421817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fcde3-dad1-4921-af20-cce10f74f063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:08.423535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fcde3-dad1-4921-af20-cce10f74f063 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:08.429538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 47/91] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:08 2022] GET /identity/v3/role_assignments?user.id=0c47d47164cb4842b0a2341a28ea82f3&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=a7f0f98b4677401690c354299b049892 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:08.437675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:08.437873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fcde3-dad1-4921-af20-cce10f74f063 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:08.464544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fde1eb-d7e5-48bc-9f2f-22e8413a8490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:08.466454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fde1eb-d7e5-48bc-9f2f-22e8413a8490 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:08.500283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:08.647481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fcde3-dad1-4921-af20-cce10f74f063 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:08.662865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 45/92] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:08.671565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fde1eb-d7e5-48bc-9f2f-22e8413a8490 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:08.697295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fae99-2f45-43ab-810f-f7950ca14078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:08.698552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fae99-2f45-43ab-810f-f7950ca14078 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:08.699588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fae99-2f45-43ab-810f-f7950ca14078 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:08.700015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fae99-2f45-43ab-810f-f7950ca14078 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:08.701353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485fae99-2f45-43ab-810f-f7950ca14078 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:08.703099 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485fae99-2f45-43ab-810f-f7950ca14078 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:08.705237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 48/93] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:08 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:08.706785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fae99-2f45-43ab-810f-f7950ca14078 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:08.820723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fae99-2f45-43ab-810f-f7950ca14078 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:08.841196 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:08.861976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aed4c9-7e47-4a15-9a1c-3233b15399c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:08.862874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aed4c9-7e47-4a15-9a1c-3233b15399c6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:08.863335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aed4c9-7e47-4a15-9a1c-3233b15399c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:08.863758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aed4c9-7e47-4a15-9a1c-3233b15399c6 None admin] PATH_INFO: `/v3/projects/2f960a38bb5245d9af199b3ca1376f2b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:08.865130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aed4c9-7e47-4a15-9a1c-3233b15399c6 None admin] RBAC: Authorizing `identity:get_project(project_id=2f960a38bb5245d9af199b3ca1376f2b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:08.866755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aed4c9-7e47-4a15-9a1c-3233b15399c6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:08.869483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:08.870365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 49/94] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:08 2022] GET /identity/v3/projects/2f960a38bb5245d9af199b3ca1376f2b => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:08.871898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aed4c9-7e47-4a15-9a1c-3233b15399c6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:08.955247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fde1eb-d7e5-48bc-9f2f-22e8413a8490 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:08.991655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aed4c9-7e47-4a15-9a1c-3233b15399c6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:09.031819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399735d9-f72e-4267-886a-53956763599a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:09.032741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399735d9-f72e-4267-886a-53956763599a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:09.033523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399735d9-f72e-4267-886a-53956763599a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:09.034027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399735d9-f72e-4267-886a-53956763599a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:09.035263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399735d9-f72e-4267-886a-53956763599a None admin] RBAC: Adding query filter params (role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=c61f47242a0d40f59b342520453f190d, user.id=0c47d47164cb4842b0a2341a28ea82f3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:09.036218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399735d9-f72e-4267-886a-53956763599a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:09.038445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399735d9-f72e-4267-886a-53956763599a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:09.045194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0e4e3-4d8e-4c10-9d62-7d24df134304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:09.045903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0e4e3-4d8e-4c10-9d62-7d24df134304 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:09.046329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0e4e3-4d8e-4c10-9d62-7d24df134304 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:09.046679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0e4e3-4d8e-4c10-9d62-7d24df134304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:09.046995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 50/95] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:08 2022] GET /identity/v3/role_assignments?user.id=0c47d47164cb4842b0a2341a28ea82f3&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=048908af57db471987eb7f2ec21e0304 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:09.047604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0e4e3-4d8e-4c10-9d62-7d24df134304 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=c61f47242a0d40f59b342520453f190d, user.id=0ba95fb65871415ca38c574ddd6396b8) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:09.047982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0e4e3-4d8e-4c10-9d62-7d24df134304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:09.048523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399735d9-f72e-4267-886a-53956763599a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:09.049685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0e4e3-4d8e-4c10-9d62-7d24df134304 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:09.059194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 46/96] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:08 2022] GET /identity/v3/role_assignments?user.id=0ba95fb65871415ca38c574ddd6396b8&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 259 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:09.092781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:09.210284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399735d9-f72e-4267-886a-53956763599a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:09.261081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921608d-1cd6-474b-b83b-57ef8719b490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:09.262298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921608d-1cd6-474b-b83b-57ef8719b490 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:09.262588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921608d-1cd6-474b-b83b-57ef8719b490 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:09.262865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921608d-1cd6-474b-b83b-57ef8719b490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:09.263613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921608d-1cd6-474b-b83b-57ef8719b490 None admin] RBAC: Adding query filter params (role.id=a7f0f98b4677401690c354299b049892, scope.project.id=2f960a38bb5245d9af199b3ca1376f2b, user.id=0c47d47164cb4842b0a2341a28ea82f3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:09.263924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921608d-1cd6-474b-b83b-57ef8719b490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:09.265455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921608d-1cd6-474b-b83b-57ef8719b490 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:09.277888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 51/97] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:09 2022] GET /identity/v3/role_assignments?user.id=0c47d47164cb4842b0a2341a28ea82f3&scope.project.id=2f960a38bb5245d9af199b3ca1376f2b&role.id=a7f0f98b4677401690c354299b049892 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:10.181770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0b4c2-89c2-4c3d-8a9e-62dfe3acd390 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:10.182275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0b4c2-89c2-4c3d-8a9e-62dfe3acd390 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:10.182349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0b4c2-89c2-4c3d-8a9e-62dfe3acd390 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:10.184430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 47/98] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:10.197611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa595036-8892-4006-8807-3792883e0747 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:10.198024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa595036-8892-4006-8807-3792883e0747 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:10.198191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa595036-8892-4006-8807-3792883e0747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:10.225383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fa595036-8892-4006-8807-3792883e0747 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:10.226614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fa595036-8892-4006-8807-3792883e0747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:10.322400 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 52/99] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:10.350331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66206732-be0b-4226-ab84-f03dfe47fe80 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:10.350641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66206732-be0b-4226-ab84-f03dfe47fe80 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:10.350899 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66206732-be0b-4226-ab84-f03dfe47fe80 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:10.352262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 48/100] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:10.358372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eae257-df77-4c2d-874f-ddb1d2f78415 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:10.358668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eae257-df77-4c2d-874f-ddb1d2f78415 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:10.358950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eae257-df77-4c2d-874f-ddb1d2f78415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:10.389437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e7eae257-df77-4c2d-874f-ddb1d2f78415 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:10.390681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7eae257-df77-4c2d-874f-ddb1d2f78415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:10.414853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:10.479124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 53/101] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:10.683052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66206732-be0b-4226-ab84-f03dfe47fe80 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:10.825581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66206732-be0b-4226-ab84-f03dfe47fe80 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:10.874676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85678355-7312-43f9-b6fe-e766e4ec9ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:10.875306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85678355-7312-43f9-b6fe-e766e4ec9ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:10.875596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85678355-7312-43f9-b6fe-e766e4ec9ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:10.875873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85678355-7312-43f9-b6fe-e766e4ec9ee7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:10.876996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85678355-7312-43f9-b6fe-e766e4ec9ee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:10.878456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85678355-7312-43f9-b6fe-e766e4ec9ee7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:10.880187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 49/102] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:10.885560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eae257-df77-4c2d-874f-ddb1d2f78415 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:10.989143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eae257-df77-4c2d-874f-ddb1d2f78415 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:11.024373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce7b7-7732-495b-9a9a-be89de487a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:11.025018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce7b7-7732-495b-9a9a-be89de487a8d None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:11.025553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce7b7-7732-495b-9a9a-be89de487a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:11.025863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcce7b7-7732-495b-9a9a-be89de487a8d None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:11.026499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce7b7-7732-495b-9a9a-be89de487a8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:11.028286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcce7b7-7732-495b-9a9a-be89de487a8d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:11.066962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85678355-7312-43f9-b6fe-e766e4ec9ee7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:11.080127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:11.083715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 54/103] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:09:10 2022] POST /identity/v3/projects => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:09:11.198031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85678355-7312-43f9-b6fe-e766e4ec9ee7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:11.246859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbc8b7-8408-44da-9e4c-0c42deaf36a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:11.247508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbc8b7-8408-44da-9e4c-0c42deaf36a3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:11.247796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbc8b7-8408-44da-9e4c-0c42deaf36a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:11.248072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbc8b7-8408-44da-9e4c-0c42deaf36a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:11.269393 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbc8b7-8408-44da-9e4c-0c42deaf36a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-28dbc8b7-8408-44da-9e4c-0c42deaf36a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:11.274850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:11.275963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:11.276785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 50/104] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:11.282501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce7b7-7732-495b-9a9a-be89de487a8d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:11.286608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:11.421723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcce7b7-7732-495b-9a9a-be89de487a8d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:11.434481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd483c0-f77e-4acf-b8df-b7f09b048443 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:11.435150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd483c0-f77e-4acf-b8df-b7f09b048443 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:11.435435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd483c0-f77e-4acf-b8df-b7f09b048443 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:11.437706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 51/105] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:11.442707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88593b99-6ae1-406d-85fe-2fbd612c81a2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:11.443226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88593b99-6ae1-406d-85fe-2fbd612c81a2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:11.443700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88593b99-6ae1-406d-85fe-2fbd612c81a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:11.473483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-88593b99-6ae1-406d-85fe-2fbd612c81a2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:11.476422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-88593b99-6ae1-406d-85fe-2fbd612c81a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:11.484947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4169b6-a46d-40ff-bf9a-b0a529c7927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:11.485850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4169b6-a46d-40ff-bf9a-b0a529c7927c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:11.486490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4169b6-a46d-40ff-bf9a-b0a529c7927c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:11.486969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4169b6-a46d-40ff-bf9a-b0a529c7927c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:11.492128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4169b6-a46d-40ff-bf9a-b0a529c7927c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:11.492693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4169b6-a46d-40ff-bf9a-b0a529c7927c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:11.494358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4169b6-a46d-40ff-bf9a-b0a529c7927c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:11.514383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 55/106] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:11.519474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4169b6-a46d-40ff-bf9a-b0a529c7927c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:11.611430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:11.625688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:11.627955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 52/107] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:11.678823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4169b6-a46d-40ff-bf9a-b0a529c7927c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:11.718358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e942f7-0a0f-4156-bfc4-25ed02f3fc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:11.719008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e942f7-0a0f-4156-bfc4-25ed02f3fc7d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:11.719293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e942f7-0a0f-4156-bfc4-25ed02f3fc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:11.719576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e942f7-0a0f-4156-bfc4-25ed02f3fc7d None admin] PATH_INFO: `/v3/users/501308170bfe4559b48e10facb420a99` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:11.734482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e942f7-0a0f-4156-bfc4-25ed02f3fc7d None admin] RBAC: Authorizing `identity:get_user(user_id=501308170bfe4559b48e10facb420a99)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:11.735974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e942f7-0a0f-4156-bfc4-25ed02f3fc7d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:11.741487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 56/108] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:11 2022] GET /identity/v3/users/501308170bfe4559b48e10facb420a99 => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:11.744095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88593b99-6ae1-406d-85fe-2fbd612c81a2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:11.855895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88593b99-6ae1-406d-85fe-2fbd612c81a2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:11.894215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1586cf5-febe-4b80-b57d-9153eef11aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:11.894842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1586cf5-febe-4b80-b57d-9153eef11aca None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:11.895129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1586cf5-febe-4b80-b57d-9153eef11aca None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:11.895408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1586cf5-febe-4b80-b57d-9153eef11aca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:11.951072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a80da1-b625-4011-8c02-4e457b698c53 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:11.951251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a80da1-b625-4011-8c02-4e457b698c53 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:11.951529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a80da1-b625-4011-8c02-4e457b698c53 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:11.952936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 57/109] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:11.958582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6ebf-d710-460a-9208-2be75be5c315 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:11.958932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6ebf-d710-460a-9208-2be75be5c315 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:11.959220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6ebf-d710-460a-9208-2be75be5c315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:11.985310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:11.991149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0cbe6ebf-d710-460a-9208-2be75be5c315 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:11.992437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0cbe6ebf-d710-460a-9208-2be75be5c315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:11.998048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1586cf5-febe-4b80-b57d-9153eef11aca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:12.000552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1586cf5-febe-4b80-b57d-9153eef11aca None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:12.139089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 53/110] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:12.143549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1586cf5-febe-4b80-b57d-9153eef11aca None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:12.150998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 58/111] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:12.155831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6ebf-d710-460a-9208-2be75be5c315 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:12.259322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:12.265609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:12.268566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1586cf5-febe-4b80-b57d-9153eef11aca None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:12.314897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88d63-6321-4f6d-85ff-bbaf43565ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:12.315545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88d63-6321-4f6d-85ff-bbaf43565ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.315833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88d63-6321-4f6d-85ff-bbaf43565ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.316118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88d63-6321-4f6d-85ff-bbaf43565ff9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.318534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6ebf-d710-460a-9208-2be75be5c315 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:12.328620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae88d63-6321-4f6d-85ff-bbaf43565ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-cae88d63-6321-4f6d-85ff-bbaf43565ff9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:12.335473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:12.336504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:12.337395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 54/112] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:12.343952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd74ee-bd38-4958-8193-8a5034700c8e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.343952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd74ee-bd38-4958-8193-8a5034700c8e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.343952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd74ee-bd38-4958-8193-8a5034700c8e None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.343952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 55/113] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:12.343952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51bc7b-32d3-4ec6-866e-fd4665f6f656 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.343952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51bc7b-32d3-4ec6-866e-fd4665f6f656 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.344382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51bc7b-32d3-4ec6-866e-fd4665f6f656 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.346053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 56/114] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:12.347070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51bc7b-32d3-4ec6-866e-fd4665f6f656 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:12.369005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6647ae-fbc8-4a2b-936b-63c4050a4b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:12.369947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6647ae-fbc8-4a2b-936b-63c4050a4b5b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.370463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6647ae-fbc8-4a2b-936b-63c4050a4b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.370978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6647ae-fbc8-4a2b-936b-63c4050a4b5b None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.373464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6647ae-fbc8-4a2b-936b-63c4050a4b5b None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:12.375213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6647ae-fbc8-4a2b-936b-63c4050a4b5b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:12.377620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 59/115] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:12 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:12.379208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87daee-4991-4810-b5cb-73fb04b9cb6a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.379707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87daee-4991-4810-b5cb-73fb04b9cb6a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.380146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87daee-4991-4810-b5cb-73fb04b9cb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.399497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bf87daee-4991-4810-b5cb-73fb04b9cb6a None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:12.401527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf87daee-4991-4810-b5cb-73fb04b9cb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:12.463708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51bc7b-32d3-4ec6-866e-fd4665f6f656 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:12.495918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 60/116] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:12.497723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628229d7-cb10-4277-a3a2-5f1610d983c2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.498075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628229d7-cb10-4277-a3a2-5f1610d983c2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.498419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628229d7-cb10-4277-a3a2-5f1610d983c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.509799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34045d38-10b0-4ceb-89b7-fb90ed564dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:12.510397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34045d38-10b0-4ceb-89b7-fb90ed564dfd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.510677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34045d38-10b0-4ceb-89b7-fb90ed564dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.510952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34045d38-10b0-4ceb-89b7-fb90ed564dfd None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.511731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34045d38-10b0-4ceb-89b7-fb90ed564dfd None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:12.512070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34045d38-10b0-4ceb-89b7-fb90ed564dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:12.513786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34045d38-10b0-4ceb-89b7-fb90ed564dfd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:12.516720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-628229d7-cb10-4277-a3a2-5f1610d983c2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:12.518281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-628229d7-cb10-4277-a3a2-5f1610d983c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:12.524350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 57/117] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:12.530528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34045d38-10b0-4ceb-89b7-fb90ed564dfd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:12.576794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:12.577753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:12.604854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 61/118] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:12.606332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628229d7-cb10-4277-a3a2-5f1610d983c2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:12.649128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34045d38-10b0-4ceb-89b7-fb90ed564dfd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:12.712085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a8d2-aa8a-4118-afe8-177ff37ea9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:12.712784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a8d2-aa8a-4118-afe8-177ff37ea9ec None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.713117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a8d2-aa8a-4118-afe8-177ff37ea9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.713400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a8d2-aa8a-4118-afe8-177ff37ea9ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.714234 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199a8d2-aa8a-4118-afe8-177ff37ea9ec None admin] RBAC: Adding query filter params (role.id=12045ab231c24ebeb9dc0c0b7f6af56a, scope.project.id=c61f47242a0d40f59b342520453f190d, user.id=501308170bfe4559b48e10facb420a99) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:12.714542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199a8d2-aa8a-4118-afe8-177ff37ea9ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:12.724883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199a8d2-aa8a-4118-afe8-177ff37ea9ec None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:12.734554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628229d7-cb10-4277-a3a2-5f1610d983c2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:12.746235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 58/119] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:12 2022] GET /identity/v3/role_assignments?user.id=501308170bfe4559b48e10facb420a99&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=12045ab231c24ebeb9dc0c0b7f6af56a => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:12.747461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a8d2-aa8a-4118-afe8-177ff37ea9ec None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:12.780951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2512-8bcf-4709-8911-a45994bccbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:12.781957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2512-8bcf-4709-8911-a45994bccbcd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.782436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2512-8bcf-4709-8911-a45994bccbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.782976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2512-8bcf-4709-8911-a45994bccbcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.814789 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d2512-8bcf-4709-8911-a45994bccbcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d59d2512-8bcf-4709-8911-a45994bccbcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:12.819578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:12.820453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:12.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 62/120] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:09:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:12.821832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2512-8bcf-4709-8911-a45994bccbcd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:12.865683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:12.884864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:12.889825 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a8d2-aa8a-4118-afe8-177ff37ea9ec None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:12.931423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffeb62-52fb-4808-9d6c-891089cc3172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:12.932072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffeb62-52fb-4808-9d6c-891089cc3172 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:12.932358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffeb62-52fb-4808-9d6c-891089cc3172 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:12.932667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffeb62-52fb-4808-9d6c-891089cc3172 None admin] PATH_INFO: `/v3/users/0c47d47164cb4842b0a2341a28ea82f3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:12.934062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffeb62-52fb-4808-9d6c-891089cc3172 None admin] RBAC: Authorizing `identity:get_user(user_id=0c47d47164cb4842b0a2341a28ea82f3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:12.935588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffeb62-52fb-4808-9d6c-891089cc3172 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:12.937506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 59/121] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:12 2022] GET /identity/v3/users/0c47d47164cb4842b0a2341a28ea82f3 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:12.938308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffeb62-52fb-4808-9d6c-891089cc3172 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:12.997072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2512-8bcf-4709-8911-a45994bccbcd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.035631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636df06-a302-42f9-bcc5-658b567ea66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.036307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636df06-a302-42f9-bcc5-658b567ea66a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.036783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636df06-a302-42f9-bcc5-658b567ea66a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.037129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636df06-a302-42f9-bcc5-658b567ea66a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.048880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636df06-a302-42f9-bcc5-658b567ea66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c636df06-a302-42f9-bcc5-658b567ea66a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:13.053633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:13.054477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:13.055266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 63/122] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:13.055892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636df06-a302-42f9-bcc5-658b567ea66a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.057153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffeb62-52fb-4808-9d6c-891089cc3172 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.100943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ae3c8-d54a-4313-9a6a-3b95b91c3251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.102246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae3c8-d54a-4313-9a6a-3b95b91c3251 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.103054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae3c8-d54a-4313-9a6a-3b95b91c3251 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.103559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ae3c8-d54a-4313-9a6a-3b95b91c3251 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.104716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ae3c8-d54a-4313-9a6a-3b95b91c3251 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:13.105223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ae3c8-d54a-4313-9a6a-3b95b91c3251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:13.107335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ae3c8-d54a-4313-9a6a-3b95b91c3251 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:13.118195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 60/123] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:13.119334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ae3c8-d54a-4313-9a6a-3b95b91c3251 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.142659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:13.173885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636df06-a302-42f9-bcc5-658b567ea66a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.175846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:13.218490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c541a-1222-4220-ba10-4543002d0fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.219364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c541a-1222-4220-ba10-4543002d0fac None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.219946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c541a-1222-4220-ba10-4543002d0fac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.220367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c541a-1222-4220-ba10-4543002d0fac None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.222464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c541a-1222-4220-ba10-4543002d0fac None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:13.224098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c541a-1222-4220-ba10-4543002d0fac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:13.225904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 64/124] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:13.226896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c541a-1222-4220-ba10-4543002d0fac None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.255728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ae3c8-d54a-4313-9a6a-3b95b91c3251 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.293982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12718b99-4de4-4899-bda5-dc49e7b0cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.295542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718b99-4de4-4899-bda5-dc49e7b0cf62 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.296233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718b99-4de4-4899-bda5-dc49e7b0cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.297042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718b99-4de4-4899-bda5-dc49e7b0cf62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.309553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12718b99-4de4-4899-bda5-dc49e7b0cf62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-12718b99-4de4-4899-bda5-dc49e7b0cf62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:13.315030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:13.315971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:13.316800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 61/125] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:13.317434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12718b99-4de4-4899-bda5-dc49e7b0cf62 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.352019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c541a-1222-4220-ba10-4543002d0fac None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.389993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b61a40-3f40-4140-8eaf-9ec555754f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.390896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b61a40-3f40-4140-8eaf-9ec555754f41 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.391319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b61a40-3f40-4140-8eaf-9ec555754f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.391731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b61a40-3f40-4140-8eaf-9ec555754f41 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.392571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b61a40-3f40-4140-8eaf-9ec555754f41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:13.393039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b61a40-3f40-4140-8eaf-9ec555754f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:13.394575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b61a40-3f40-4140-8eaf-9ec555754f41 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:13.398795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:13.404621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 65/126] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:13.405788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b61a40-3f40-4140-8eaf-9ec555754f41 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.433970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12718b99-4de4-4899-bda5-dc49e7b0cf62 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.442708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:13.471162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3fa20-b153-4078-830d-b55d0f863830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.471813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3fa20-b153-4078-830d-b55d0f863830 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.472150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3fa20-b153-4078-830d-b55d0f863830 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.472460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3fa20-b153-4078-830d-b55d0f863830 None admin] PATH_INFO: `/v3/users/0ba95fb65871415ca38c574ddd6396b8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.473608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3fa20-b153-4078-830d-b55d0f863830 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba95fb65871415ca38c574ddd6396b8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:13.475115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3fa20-b153-4078-830d-b55d0f863830 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:13.476964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 62/127] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/users/0ba95fb65871415ca38c574ddd6396b8 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:13.477828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3fa20-b153-4078-830d-b55d0f863830 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.527565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b61a40-3f40-4140-8eaf-9ec555754f41 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.572678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca370b6e-9d18-4869-96d7-0d92d3120181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.573280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca370b6e-9d18-4869-96d7-0d92d3120181 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.573842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca370b6e-9d18-4869-96d7-0d92d3120181 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.574782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca370b6e-9d18-4869-96d7-0d92d3120181 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d/users/0c47d47164cb4842b0a2341a28ea82f3/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.577711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca370b6e-9d18-4869-96d7-0d92d3120181 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61f47242a0d40f59b342520453f190d, user_id=0c47d47164cb4842b0a2341a28ea82f3, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:13.579221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca370b6e-9d18-4869-96d7-0d92d3120181 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:13.599350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 66/128] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:09:13 2022] PUT /identity/v3/projects/c61f47242a0d40f59b342520453f190d/users/0c47d47164cb4842b0a2341a28ea82f3/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:09:13.600361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca370b6e-9d18-4869-96d7-0d92d3120181 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.627223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3fa20-b153-4078-830d-b55d0f863830 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.708555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb9819-73a6-4e7f-89de-35703d64774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.709184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb9819-73a6-4e7f-89de-35703d64774b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.709467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb9819-73a6-4e7f-89de-35703d64774b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.709842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb9819-73a6-4e7f-89de-35703d64774b None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.710534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb9819-73a6-4e7f-89de-35703d64774b None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:13.710940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb9819-73a6-4e7f-89de-35703d64774b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:13.712354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb9819-73a6-4e7f-89de-35703d64774b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:13.732790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca370b6e-9d18-4869-96d7-0d92d3120181 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.742454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 63/129] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:13.743327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb9819-73a6-4e7f-89de-35703d64774b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.754858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:13.756366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:13.783475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592b0f8-6d51-4160-80e4-3a653ad44991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.784748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592b0f8-6d51-4160-80e4-3a653ad44991 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.785040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592b0f8-6d51-4160-80e4-3a653ad44991 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.785433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592b0f8-6d51-4160-80e4-3a653ad44991 None admin] PATH_INFO: `/v3/users/0c47d47164cb4842b0a2341a28ea82f3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.786696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592b0f8-6d51-4160-80e4-3a653ad44991 None admin] RBAC: Authorizing `identity:get_user(user_id=0c47d47164cb4842b0a2341a28ea82f3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:13.788277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592b0f8-6d51-4160-80e4-3a653ad44991 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:13.790271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 67/130] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/users/0c47d47164cb4842b0a2341a28ea82f3 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:13.791285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592b0f8-6d51-4160-80e4-3a653ad44991 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.921651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb9819-73a6-4e7f-89de-35703d64774b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.924162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592b0f8-6d51-4160-80e4-3a653ad44991 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:13.962891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fc1fc-3a16-4bfd-a7a8-383c37b3174e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.963637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fc1fc-3a16-4bfd-a7a8-383c37b3174e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.963922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fc1fc-3a16-4bfd-a7a8-383c37b3174e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.964235 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fc1fc-3a16-4bfd-a7a8-383c37b3174e None admin] PATH_INFO: `/v3/users/0c47d47164cb4842b0a2341a28ea82f3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.965188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463e543-5f5f-43d7-84b5-702c4bb87057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:13.966015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463e543-5f5f-43d7-84b5-702c4bb87057 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.966121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fc1fc-3a16-4bfd-a7a8-383c37b3174e None admin] RBAC: Authorizing `identity:get_user(user_id=0c47d47164cb4842b0a2341a28ea82f3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:13.966295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463e543-5f5f-43d7-84b5-702c4bb87057 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.966579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463e543-5f5f-43d7-84b5-702c4bb87057 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.967671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fc1fc-3a16-4bfd-a7a8-383c37b3174e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:13.967775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463e543-5f5f-43d7-84b5-702c4bb87057 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:13.969287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463e543-5f5f-43d7-84b5-702c4bb87057 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:13.969473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 68/131] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/users/0c47d47164cb4842b0a2341a28ea82f3 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:13.971459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 64/132] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:13.972501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c248-d3f4-407a-9a76-007824fb94b7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:13.972790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c248-d3f4-407a-9a76-007824fb94b7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:13.973068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225c248-d3f4-407a-9a76-007824fb94b7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:13.974763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fc1fc-3a16-4bfd-a7a8-383c37b3174e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:13.979223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 65/133] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:13.980056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225c248-d3f4-407a-9a76-007824fb94b7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:14.014460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:14.078179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:14.099569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fc1fc-3a16-4bfd-a7a8-383c37b3174e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:14.122695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225c248-d3f4-407a-9a76-007824fb94b7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:14.138830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f458c-73e7-4037-8f2b-743643f40324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:14.139484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f458c-73e7-4037-8f2b-743643f40324 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:14.139882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f458c-73e7-4037-8f2b-743643f40324 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:14.140171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f458c-73e7-4037-8f2b-743643f40324 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:14.141660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f458c-73e7-4037-8f2b-743643f40324 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:14.143316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f458c-73e7-4037-8f2b-743643f40324 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:14.145115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 69/134] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:14.146014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f458c-73e7-4037-8f2b-743643f40324 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:14.173771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f238a-921c-40f2-91d9-ea2d553bc535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:14.174696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f238a-921c-40f2-91d9-ea2d553bc535 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:14.175029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f238a-921c-40f2-91d9-ea2d553bc535 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:14.175618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f238a-921c-40f2-91d9-ea2d553bc535 None admin] PATH_INFO: `/v3/projects/2f960a38bb5245d9af199b3ca1376f2b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:14.177450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f238a-921c-40f2-91d9-ea2d553bc535 None admin] RBAC: Authorizing `identity:get_project(project_id=2f960a38bb5245d9af199b3ca1376f2b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:14.179379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f238a-921c-40f2-91d9-ea2d553bc535 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:14.182070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 66/135] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:13 2022] GET /identity/v3/projects/2f960a38bb5245d9af199b3ca1376f2b => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:14.183094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab4922-babc-435a-aba2-ea183c51103f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:14.183404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab4922-babc-435a-aba2-ea183c51103f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:14.183684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab4922-babc-435a-aba2-ea183c51103f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:14.204923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-70ab4922-babc-435a-aba2-ea183c51103f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:14.206140 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-70ab4922-babc-435a-aba2-ea183c51103f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:14.269895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:14.278077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f458c-73e7-4037-8f2b-743643f40324 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:14.278335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 67/136] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:14.285878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab4922-babc-435a-aba2-ea183c51103f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:14.347222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5ed94-a0c9-4c67-bd96-99ff79973099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:14.347953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5ed94-a0c9-4c67-bd96-99ff79973099 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:14.348296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5ed94-a0c9-4c67-bd96-99ff79973099 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:14.348602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5ed94-a0c9-4c67-bd96-99ff79973099 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d/users/0ba95fb65871415ca38c574ddd6396b8/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:14.356963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5ed94-a0c9-4c67-bd96-99ff79973099 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61f47242a0d40f59b342520453f190d, user_id=0ba95fb65871415ca38c574ddd6396b8, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:14.360112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5ed94-a0c9-4c67-bd96-99ff79973099 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:14.385131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 70/137] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:09:14 2022] PUT /identity/v3/projects/c61f47242a0d40f59b342520453f190d/users/0ba95fb65871415ca38c574ddd6396b8/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:09:14.386299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5ed94-a0c9-4c67-bd96-99ff79973099 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:14.412105 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:14.479315 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab4922-babc-435a-aba2-ea183c51103f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:14.579619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fb683-9510-4475-8b13-cae6307f60df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:14.580300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fb683-9510-4475-8b13-cae6307f60df None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:14.580611 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fb683-9510-4475-8b13-cae6307f60df None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:14.581089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fb683-9510-4475-8b13-cae6307f60df None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d/users/0c47d47164cb4842b0a2341a28ea82f3/roles/048908af57db471987eb7f2ec21e0304` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:14.585335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fb683-9510-4475-8b13-cae6307f60df None admin] RBAC: Authorizing `identity:create_grant(project_id=c61f47242a0d40f59b342520453f190d, user_id=0c47d47164cb4842b0a2341a28ea82f3, role_id=048908af57db471987eb7f2ec21e0304)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:14.586825 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fb683-9510-4475-8b13-cae6307f60df None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:14.615004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5ed94-a0c9-4c67-bd96-99ff79973099 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:14.619708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 68/138] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:09:14 2022] PUT /identity/v3/projects/c61f47242a0d40f59b342520453f190d/users/0c47d47164cb4842b0a2341a28ea82f3/roles/048908af57db471987eb7f2ec21e0304 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:09:14.668192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe71ac9-3e8e-4e0e-9213-120b75330c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:14.671486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe71ac9-3e8e-4e0e-9213-120b75330c02 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:14.671486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe71ac9-3e8e-4e0e-9213-120b75330c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:14.671486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe71ac9-3e8e-4e0e-9213-120b75330c02 None admin] PATH_INFO: `/v3/projects/2f960a38bb5245d9af199b3ca1376f2b/users/0c47d47164cb4842b0a2341a28ea82f3/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:14.674404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe71ac9-3e8e-4e0e-9213-120b75330c02 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f960a38bb5245d9af199b3ca1376f2b, user_id=0c47d47164cb4842b0a2341a28ea82f3, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:14.675769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe71ac9-3e8e-4e0e-9213-120b75330c02 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:14.696312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 71/139] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:09:14 2022] PUT /identity/v3/projects/2f960a38bb5245d9af199b3ca1376f2b/users/0c47d47164cb4842b0a2341a28ea82f3/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:09:14.733895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fb683-9510-4475-8b13-cae6307f60df None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:14.883636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fb683-9510-4475-8b13-cae6307f60df None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:14.914902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:14.924096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb72f3c-1332-4791-a176-d318b2f7b478 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:14.924764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb72f3c-1332-4791-a176-d318b2f7b478 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:14.925268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb72f3c-1332-4791-a176-d318b2f7b478 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:14.926881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 72/140] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:14.931204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a735ae-a9ce-4987-aee4-e22bbc23fb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:14.931204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a735ae-a9ce-4987-aee4-e22bbc23fb46 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:14.931204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a735ae-a9ce-4987-aee4-e22bbc23fb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:14.931204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a735ae-a9ce-4987-aee4-e22bbc23fb46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:14.931562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a735ae-a9ce-4987-aee4-e22bbc23fb46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:14.934456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a735ae-a9ce-4987-aee4-e22bbc23fb46 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:14.934926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 69/141] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:14.936846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a673ab0-6a3b-4b4f-8f81-8666a9fe9373 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:14.937589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a673ab0-6a3b-4b4f-8f81-8666a9fe9373 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:14.938063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a673ab0-6a3b-4b4f-8f81-8666a9fe9373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:14.941445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a735ae-a9ce-4987-aee4-e22bbc23fb46 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:14.982140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7a673ab0-6a3b-4b4f-8f81-8666a9fe9373 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:14.983803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7a673ab0-6a3b-4b4f-8f81-8666a9fe9373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:15.002462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:15.060722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a735ae-a9ce-4987-aee4-e22bbc23fb46 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:15.103807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2267b04-52a5-4d3e-9c79-439c568ea34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:15.104444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2267b04-52a5-4d3e-9c79-439c568ea34a None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:15.104728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2267b04-52a5-4d3e-9c79-439c568ea34a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:15.105004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2267b04-52a5-4d3e-9c79-439c568ea34a None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:15.105720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2267b04-52a5-4d3e-9c79-439c568ea34a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:15.107894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2267b04-52a5-4d3e-9c79-439c568ea34a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:15.137853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 73/142] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:15.146528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 70/143] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:09:14 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:09:15.513107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a673ab0-6a3b-4b4f-8f81-8666a9fe9373 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:15.631621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a673ab0-6a3b-4b4f-8f81-8666a9fe9373 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:15.684415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e721394-c4dd-475c-8390-700e70634532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:15.684415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e721394-c4dd-475c-8390-700e70634532 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:15.684415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e721394-c4dd-475c-8390-700e70634532 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:15.684415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e721394-c4dd-475c-8390-700e70634532 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:15.702840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e721394-c4dd-475c-8390-700e70634532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8e721394-c4dd-475c-8390-700e70634532 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:15.708228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:15.709970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:15.711281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:15.715012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 74/144] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:15.716261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2267b04-52a5-4d3e-9c79-439c568ea34a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:15.798987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:15.860528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2267b04-52a5-4d3e-9c79-439c568ea34a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:15.906112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ace17-98d4-4975-a718-9e7ad785657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:15.906779 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ace17-98d4-4975-a718-9e7ad785657e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:15.907064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ace17-98d4-4975-a718-9e7ad785657e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:15.907340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ace17-98d4-4975-a718-9e7ad785657e None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:15.908017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ace17-98d4-4975-a718-9e7ad785657e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:15.908325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ace17-98d4-4975-a718-9e7ad785657e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:15.909762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ace17-98d4-4975-a718-9e7ad785657e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:15.923954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 71/145] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:15.929517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e721394-c4dd-475c-8390-700e70634532 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:15.943325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:16.069127 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e721394-c4dd-475c-8390-700e70634532 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:16.110985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca007a-21c5-46b5-82d7-8f8c1defee07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:16.111632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca007a-21c5-46b5-82d7-8f8c1defee07 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:16.111916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca007a-21c5-46b5-82d7-8f8c1defee07 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:16.112193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca007a-21c5-46b5-82d7-8f8c1defee07 None admin] PATH_INFO: `/v3/users/501308170bfe4559b48e10facb420a99` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:16.113395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca007a-21c5-46b5-82d7-8f8c1defee07 None admin] RBAC: Authorizing `identity:get_user(user_id=501308170bfe4559b48e10facb420a99)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:16.114946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca007a-21c5-46b5-82d7-8f8c1defee07 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:16.117551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 75/146] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:15 2022] GET /identity/v3/users/501308170bfe4559b48e10facb420a99 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:16.122689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ace17-98d4-4975-a718-9e7ad785657e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:16.128157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a782b0b-5b92-4e68-aa55-99c09af720ba None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:16.128471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a782b0b-5b92-4e68-aa55-99c09af720ba None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:16.128753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a782b0b-5b92-4e68-aa55-99c09af720ba None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:16.130076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 76/147] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:09:16.143234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873c19-ae61-401b-9744-a5dccd737cf9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:16.143525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873c19-ae61-401b-9744-a5dccd737cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:16.143805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2873c19-ae61-401b-9744-a5dccd737cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:16.167555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f2873c19-ae61-401b-9744-a5dccd737cf9 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:16.169592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f2873c19-ae61-401b-9744-a5dccd737cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:16.253325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 77/148] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:16.291256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ace17-98d4-4975-a718-9e7ad785657e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:16.337310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:16.339933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7ea62-0518-4293-8d81-b667a3e95d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:16.340603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7ea62-0518-4293-8d81-b667a3e95d62 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:16.340892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7ea62-0518-4293-8d81-b667a3e95d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:16.341172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7ea62-0518-4293-8d81-b667a3e95d62 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:16.343435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7ea62-0518-4293-8d81-b667a3e95d62 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:16.344983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7ea62-0518-4293-8d81-b667a3e95d62 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:16.347339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 72/149] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:16 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:16.352937 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873c19-ae61-401b-9744-a5dccd737cf9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:16.357213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:16.483307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2873c19-ae61-401b-9744-a5dccd737cf9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:16.524558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3dda5-9eaf-4377-bc63-07394521eb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:16.525302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3dda5-9eaf-4377-bc63-07394521eb07 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:16.525761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3dda5-9eaf-4377-bc63-07394521eb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:16.526038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3dda5-9eaf-4377-bc63-07394521eb07 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d/users/501308170bfe4559b48e10facb420a99/roles/12045ab231c24ebeb9dc0c0b7f6af56a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:16.528681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3dda5-9eaf-4377-bc63-07394521eb07 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61f47242a0d40f59b342520453f190d, user_id=501308170bfe4559b48e10facb420a99, role_id=12045ab231c24ebeb9dc0c0b7f6af56a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:16.530147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3dda5-9eaf-4377-bc63-07394521eb07 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:16.551039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 78/150] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:09:16 2022] PUT /identity/v3/projects/c61f47242a0d40f59b342520453f190d/users/501308170bfe4559b48e10facb420a99/roles/12045ab231c24ebeb9dc0c0b7f6af56a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:09:16.671539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7ea62-0518-4293-8d81-b667a3e95d62 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:16.808816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7ea62-0518-4293-8d81-b667a3e95d62 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:16.854688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83edbf3-c790-44af-90b4-a31aaaef0d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:16.855379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83edbf3-c790-44af-90b4-a31aaaef0d3c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:16.855669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83edbf3-c790-44af-90b4-a31aaaef0d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:16.855951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83edbf3-c790-44af-90b4-a31aaaef0d3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:16.870243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83edbf3-c790-44af-90b4-a31aaaef0d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f83edbf3-c790-44af-90b4-a31aaaef0d3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:16.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:16.877341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:16.878112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:16.880459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 73/151] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:16.887878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3dda5-9eaf-4377-bc63-07394521eb07 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:16.983960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:17.028578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3dda5-9eaf-4377-bc63-07394521eb07 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:17.071655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97960504-5348-4ba4-8351-8584637fbf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:17.072260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97960504-5348-4ba4-8351-8584637fbf6a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:17.072560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97960504-5348-4ba4-8351-8584637fbf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:17.072831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97960504-5348-4ba4-8351-8584637fbf6a None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:17.073502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97960504-5348-4ba4-8351-8584637fbf6a None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:17.073804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97960504-5348-4ba4-8351-8584637fbf6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:17.075865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97960504-5348-4ba4-8351-8584637fbf6a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:17.084608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 79/152] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:17.090865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83edbf3-c790-44af-90b4-a31aaaef0d3c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:17.184952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:17.257104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83edbf3-c790-44af-90b4-a31aaaef0d3c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:17.296522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4f404-7ecb-42ab-9688-ed3f10fff3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:17.297541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4f404-7ecb-42ab-9688-ed3f10fff3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:17.298046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4f404-7ecb-42ab-9688-ed3f10fff3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:17.298532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4f404-7ecb-42ab-9688-ed3f10fff3d7 None admin] PATH_INFO: `/v3/users/0c47d47164cb4842b0a2341a28ea82f3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:17.299911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4f404-7ecb-42ab-9688-ed3f10fff3d7 None admin] RBAC: Authorizing `identity:get_user(user_id=0c47d47164cb4842b0a2341a28ea82f3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:17.301647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4f404-7ecb-42ab-9688-ed3f10fff3d7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:17.303798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 74/153] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:17 2022] GET /identity/v3/users/0c47d47164cb4842b0a2341a28ea82f3 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:17.311534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97960504-5348-4ba4-8351-8584637fbf6a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:17.398300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040d480-0f19-4c34-be6d-888f004489e8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:17.398962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040d480-0f19-4c34-be6d-888f004489e8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:17.399449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040d480-0f19-4c34-be6d-888f004489e8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:17.405583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 75/154] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:17.410800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacba5d0-9486-4eb9-b4e0-dbf84c4b0993 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:17.411352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacba5d0-9486-4eb9-b4e0-dbf84c4b0993 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:17.411835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacba5d0-9486-4eb9-b4e0-dbf84c4b0993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:17.442132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97960504-5348-4ba4-8351-8584637fbf6a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:17.443438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bacba5d0-9486-4eb9-b4e0-dbf84c4b0993 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:17.444900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bacba5d0-9486-4eb9-b4e0-dbf84c4b0993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:17.476795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:17.478577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429156d7-d224-4f56-848c-2bec8c93c4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:17.479191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429156d7-d224-4f56-848c-2bec8c93c4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:17.479505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429156d7-d224-4f56-848c-2bec8c93c4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:17.479777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429156d7-d224-4f56-848c-2bec8c93c4b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:17.553191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429156d7-d224-4f56-848c-2bec8c93c4b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:17.554786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429156d7-d224-4f56-848c-2bec8c93c4b2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:17.590362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 76/155] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:17.659112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 80/156] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:17.664686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacba5d0-9486-4eb9-b4e0-dbf84c4b0993 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:17.711114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:17.792210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacba5d0-9486-4eb9-b4e0-dbf84c4b0993 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:17.840171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb38b69-da40-4699-9e1c-eba66b7cdf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:17.841159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb38b69-da40-4699-9e1c-eba66b7cdf24 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:17.841159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb38b69-da40-4699-9e1c-eba66b7cdf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:17.841419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb38b69-da40-4699-9e1c-eba66b7cdf24 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:17.842477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb38b69-da40-4699-9e1c-eba66b7cdf24 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:17.843948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb38b69-da40-4699-9e1c-eba66b7cdf24 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:17.845929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 77/157] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:17 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:17.851582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429156d7-d224-4f56-848c-2bec8c93c4b2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:17.884844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:17.959118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429156d7-d224-4f56-848c-2bec8c93c4b2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:17.996664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac180efd-492c-40ad-bf3f-29e0eb3dae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:17.997274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac180efd-492c-40ad-bf3f-29e0eb3dae95 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:17.997583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac180efd-492c-40ad-bf3f-29e0eb3dae95 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:17.997850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac180efd-492c-40ad-bf3f-29e0eb3dae95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:17.998601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac180efd-492c-40ad-bf3f-29e0eb3dae95 None admin] RBAC: Adding query filter params (role.id=a7f0f98b4677401690c354299b049892, scope.project.id=c61f47242a0d40f59b342520453f190d, user.id=0c47d47164cb4842b0a2341a28ea82f3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:17.998911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac180efd-492c-40ad-bf3f-29e0eb3dae95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:18.000449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac180efd-492c-40ad-bf3f-29e0eb3dae95 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:18.007961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 81/158] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:17 2022] GET /identity/v3/role_assignments?user.id=0c47d47164cb4842b0a2341a28ea82f3&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=a7f0f98b4677401690c354299b049892 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:18.280377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cad4b6-9b4f-4f36-8318-906df783fe0b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:18.280769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cad4b6-9b4f-4f36-8318-906df783fe0b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:18.280962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cad4b6-9b4f-4f36-8318-906df783fe0b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:18.282723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 78/159] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:18.294759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c86c9a-9c0a-4327-b63e-6e9dcea04411 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:18.295076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c86c9a-9c0a-4327-b63e-6e9dcea04411 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:18.295306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c86c9a-9c0a-4327-b63e-6e9dcea04411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:18.311216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cad4b6-9b4f-4f36-8318-906df783fe0b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:18.321868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-29c86c9a-9c0a-4327-b63e-6e9dcea04411 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:18.323091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-29c86c9a-9c0a-4327-b63e-6e9dcea04411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:18.420526 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 82/160] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:18.489333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cad4b6-9b4f-4f36-8318-906df783fe0b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:18.495497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17ebea-77b2-4bff-9b76-ba9602d51e26 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:18.495852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17ebea-77b2-4bff-9b76-ba9602d51e26 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:18.496858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17ebea-77b2-4bff-9b76-ba9602d51e26 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:18.498566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 83/161] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:18.503607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:18.508562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e61fa-62a7-4934-a087-c56efc43679c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:18.509063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e61fa-62a7-4934-a087-c56efc43679c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:18.509542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e61fa-62a7-4934-a087-c56efc43679c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:18.533831 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5a4ac-0f27-4721-ad65-9fc7a3b5653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:18.534834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5a4ac-0f27-4721-ad65-9fc7a3b5653e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:18.535375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5a4ac-0f27-4721-ad65-9fc7a3b5653e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:18.535867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5a4ac-0f27-4721-ad65-9fc7a3b5653e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:18.537900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cc8e61fa-62a7-4934-a087-c56efc43679c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:18.539946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc8e61fa-62a7-4934-a087-c56efc43679c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:18.549999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5a4ac-0f27-4721-ad65-9fc7a3b5653e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0fe5a4ac-0f27-4721-ad65-9fc7a3b5653e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:18.555261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:18.556229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:18.557610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:18.560259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 79/162] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:09:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:18.566027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5a4ac-0f27-4721-ad65-9fc7a3b5653e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:18.568775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:18.631023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 84/163] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:18.735772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5a4ac-0f27-4721-ad65-9fc7a3b5653e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:18.770992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4f1d-ce5c-45f2-bd1c-9ff6946d68c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:18.771831 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4f1d-ce5c-45f2-bd1c-9ff6946d68c8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:18.772280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4f1d-ce5c-45f2-bd1c-9ff6946d68c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:18.772731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef4f1d-ce5c-45f2-bd1c-9ff6946d68c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:18.773557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef4f1d-ce5c-45f2-bd1c-9ff6946d68c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:18.774023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef4f1d-ce5c-45f2-bd1c-9ff6946d68c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:18.775910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef4f1d-ce5c-45f2-bd1c-9ff6946d68c8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:18.785919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 80/164] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:18.791553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e61fa-62a7-4934-a087-c56efc43679c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:18.936761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e61fa-62a7-4934-a087-c56efc43679c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:18.981149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ded66-6706-47a0-afa0-0df06d324982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:18.982145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ded66-6706-47a0-afa0-0df06d324982 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:18.982638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ded66-6706-47a0-afa0-0df06d324982 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:18.983141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ded66-6706-47a0-afa0-0df06d324982 None admin] PATH_INFO: `/v3/users/0ba95fb65871415ca38c574ddd6396b8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:18.985437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ded66-6706-47a0-afa0-0df06d324982 None admin] RBAC: Authorizing `identity:get_user(user_id=0ba95fb65871415ca38c574ddd6396b8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:18.987234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ded66-6706-47a0-afa0-0df06d324982 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:18.989068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 85/165] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:18 2022] GET /identity/v3/users/0ba95fb65871415ca38c574ddd6396b8 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:18.994900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4f1d-ce5c-45f2-bd1c-9ff6946d68c8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:19.023489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ded66-6706-47a0-afa0-0df06d324982 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:19.035415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:19.052870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:19.165457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ded66-6706-47a0-afa0-0df06d324982 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:19.201275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81b092-deb2-43df-8eb9-d8e45c837a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:19.202083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81b092-deb2-43df-8eb9-d8e45c837a8b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:19.202363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81b092-deb2-43df-8eb9-d8e45c837a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:19.202547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef4f1d-ce5c-45f2-bd1c-9ff6946d68c8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:19.202661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81b092-deb2-43df-8eb9-d8e45c837a8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:19.213936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa81b092-deb2-43df-8eb9-d8e45c837a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-fa81b092-deb2-43df-8eb9-d8e45c837a8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:19.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:19.220094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:19.220901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 86/166] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:19.221803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81b092-deb2-43df-8eb9-d8e45c837a8b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:19.305855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8312a4-e040-4b7c-8527-452d10418917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:19.306402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8312a4-e040-4b7c-8527-452d10418917 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:19.306682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8312a4-e040-4b7c-8527-452d10418917 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:19.307003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8312a4-e040-4b7c-8527-452d10418917 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:19.307443 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:19.326474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81b092-deb2-43df-8eb9-d8e45c837a8b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:19.363545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ce90e-2266-4abd-85ce-f4a356036911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:19.364193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce90e-2266-4abd-85ce-f4a356036911 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:19.364497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce90e-2266-4abd-85ce-f4a356036911 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:19.364774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce90e-2266-4abd-85ce-f4a356036911 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:19.376723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ce90e-2266-4abd-85ce-f4a356036911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6d7ce90e-2266-4abd-85ce-f4a356036911 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:19.381746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:19.382632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:19.383420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 87/167] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:19.384164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ce90e-2266-4abd-85ce-f4a356036911 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:19.455237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8312a4-e040-4b7c-8527-452d10418917 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:19.456758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8312a4-e040-4b7c-8527-452d10418917 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:19.544870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ce90e-2266-4abd-85ce-f4a356036911 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:19.546196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:19.621832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecb6a2-367c-414e-8ba3-7bbd358ac181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:19.622510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecb6a2-367c-414e-8ba3-7bbd358ac181 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:19.622798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecb6a2-367c-414e-8ba3-7bbd358ac181 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:19.623069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecb6a2-367c-414e-8ba3-7bbd358ac181 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:19.623749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecb6a2-367c-414e-8ba3-7bbd358ac181 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:19.624050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecb6a2-367c-414e-8ba3-7bbd358ac181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:19.625488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecb6a2-367c-414e-8ba3-7bbd358ac181 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:19.637131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:19.644188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 88/168] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:19.645051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecb6a2-367c-414e-8ba3-7bbd358ac181 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:19.743479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 81/169] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:19.744442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8312a4-e040-4b7c-8527-452d10418917 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:19.807519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecb6a2-367c-414e-8ba3-7bbd358ac181 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:19.873256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d04588-f1e1-4e86-b22b-7d80c6f81874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:19.873796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d04588-f1e1-4e86-b22b-7d80c6f81874 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:19.874077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d04588-f1e1-4e86-b22b-7d80c6f81874 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:19.874351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d04588-f1e1-4e86-b22b-7d80c6f81874 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:19.875032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d04588-f1e1-4e86-b22b-7d80c6f81874 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:19.875336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d04588-f1e1-4e86-b22b-7d80c6f81874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:19.876768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d04588-f1e1-4e86-b22b-7d80c6f81874 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:19.885782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 89/170] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:19.887215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d04588-f1e1-4e86-b22b-7d80c6f81874 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:19.971091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8312a4-e040-4b7c-8527-452d10418917 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:19.993490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:20.031679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d04588-f1e1-4e86-b22b-7d80c6f81874 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:20.053131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:20.068750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586797f-7471-411f-a8b1-8ce935a640cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:20.068951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6d0ba-86d7-4030-b71e-271d858eeecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:20.069383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586797f-7471-411f-a8b1-8ce935a640cd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.069490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6d0ba-86d7-4030-b71e-271d858eeecf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.069655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6d0ba-86d7-4030-b71e-271d858eeecf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.069745 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586797f-7471-411f-a8b1-8ce935a640cd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.069924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6d0ba-86d7-4030-b71e-271d858eeecf None admin] PATH_INFO: `/v3/users/0c47d47164cb4842b0a2341a28ea82f3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.070033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586797f-7471-411f-a8b1-8ce935a640cd None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.071081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6d0ba-86d7-4030-b71e-271d858eeecf None admin] RBAC: Authorizing `identity:get_user(user_id=0c47d47164cb4842b0a2341a28ea82f3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:20.071196 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586797f-7471-411f-a8b1-8ce935a640cd None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:20.072570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586797f-7471-411f-a8b1-8ce935a640cd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:20.072677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6d0ba-86d7-4030-b71e-271d858eeecf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:20.075175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 90/171] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:19 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:20.075948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586797f-7471-411f-a8b1-8ce935a640cd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:20.081019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 82/172] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:19 2022] GET /identity/v3/users/0c47d47164cb4842b0a2341a28ea82f3 => generated 275 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:20.082302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127d994-1498-4002-a005-a12ba52e31de None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.082587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127d994-1498-4002-a005-a12ba52e31de None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.082907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127d994-1498-4002-a005-a12ba52e31de None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.084306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 83/173] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:20.084897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127d994-1498-4002-a005-a12ba52e31de None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:20.199705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586797f-7471-411f-a8b1-8ce935a640cd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:20.228721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127d994-1498-4002-a005-a12ba52e31de None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:20.231706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:20.238485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a319e5-ea52-4d61-a469-268addc0fd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:20.239237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a319e5-ea52-4d61-a469-268addc0fd51 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.239522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a319e5-ea52-4d61-a469-268addc0fd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.239798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a319e5-ea52-4d61-a469-268addc0fd51 None admin] PATH_INFO: `/v3/users/0c47d47164cb4842b0a2341a28ea82f3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.241117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a319e5-ea52-4d61-a469-268addc0fd51 None admin] RBAC: Authorizing `identity:get_user(user_id=0c47d47164cb4842b0a2341a28ea82f3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:20.242649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a319e5-ea52-4d61-a469-268addc0fd51 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:20.244783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 91/174] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:20 2022] GET /identity/v3/users/0c47d47164cb4842b0a2341a28ea82f3 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:20.249182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a319e5-ea52-4d61-a469-268addc0fd51 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:20.265602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b96ab-4431-48d3-99a0-84bc08dd2b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:20.266574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b96ab-4431-48d3-99a0-84bc08dd2b7c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.266862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b96ab-4431-48d3-99a0-84bc08dd2b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.267209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b96ab-4431-48d3-99a0-84bc08dd2b7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.267868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b96ab-4431-48d3-99a0-84bc08dd2b7c None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=c61f47242a0d40f59b342520453f190d, user.id=0ba95fb65871415ca38c574ddd6396b8) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:20.268176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b96ab-4431-48d3-99a0-84bc08dd2b7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:20.269739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b96ab-4431-48d3-99a0-84bc08dd2b7c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:20.284439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 84/175] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:20 2022] GET /identity/v3/role_assignments?user.id=0ba95fb65871415ca38c574ddd6396b8&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:20.284439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da0b3-d699-4489-a78b-8692a7382355 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.284439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da0b3-d699-4489-a78b-8692a7382355 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.284439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da0b3-d699-4489-a78b-8692a7382355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.301758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-988da0b3-d699-4489-a78b-8692a7382355 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:20.303085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-988da0b3-d699-4489-a78b-8692a7382355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:20.372832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:20.376358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a319e5-ea52-4d61-a469-268addc0fd51 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:20.395641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 85/176] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:20.402247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6e22f-f293-418b-9e44-0024d465556f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.402603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6e22f-f293-418b-9e44-0024d465556f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.402838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6e22f-f293-418b-9e44-0024d465556f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.408222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 86/177] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:20.412742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d18e71-c003-43b1-904c-d6c8c7f511c0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.413021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d18e71-c003-43b1-904c-d6c8c7f511c0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.413300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d18e71-c003-43b1-904c-d6c8c7f511c0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.415383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 87/178] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:20.415823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d18e71-c003-43b1-904c-d6c8c7f511c0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:20.433970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5acfb9-4054-4689-a68f-80e212caafa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:20.435024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5acfb9-4054-4689-a68f-80e212caafa4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.435578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5acfb9-4054-4689-a68f-80e212caafa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.436045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5acfb9-4054-4689-a68f-80e212caafa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.531709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5acfb9-4054-4689-a68f-80e212caafa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:20.536150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5acfb9-4054-4689-a68f-80e212caafa4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:20.556803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:20.571134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d18e71-c003-43b1-904c-d6c8c7f511c0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:20.623341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a06fdd-1f63-4740-8309-062c17914885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:20.623952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a06fdd-1f63-4740-8309-062c17914885 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.624234 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a06fdd-1f63-4740-8309-062c17914885 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.624547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a06fdd-1f63-4740-8309-062c17914885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.666045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 92/179] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:20.667105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d800df3-e2d7-43e8-9d36-a0f7d682ff15 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.667450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d800df3-e2d7-43e8-9d36-a0f7d682ff15 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.668474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d800df3-e2d7-43e8-9d36-a0f7d682ff15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.688278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1d800df3-e2d7-43e8-9d36-a0f7d682ff15 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:20.689502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1d800df3-e2d7-43e8-9d36-a0f7d682ff15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:20.705770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:20.715506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a06fdd-1f63-4740-8309-062c17914885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:20.717223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a06fdd-1f63-4740-8309-062c17914885 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:20.759042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 93/180] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:20.760289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249fae0-3b51-4383-b7e1-a97df39e86e4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.760602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249fae0-3b51-4383-b7e1-a97df39e86e4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.760885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249fae0-3b51-4383-b7e1-a97df39e86e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.792360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3249fae0-3b51-4383-b7e1-a97df39e86e4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:20.794758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3249fae0-3b51-4383-b7e1-a97df39e86e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:20.817868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 88/181] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:20.819579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11171826-4d5d-41b7-9d37-87eb37175b22 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.820110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11171826-4d5d-41b7-9d37-87eb37175b22 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.820293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11171826-4d5d-41b7-9d37-87eb37175b22 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.821517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:20.821876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 89/182] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:20.826106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11171826-4d5d-41b7-9d37-87eb37175b22 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:20.868034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 94/183] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:20.868915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249fae0-3b51-4383-b7e1-a97df39e86e4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:20.933543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:20.945058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11171826-4d5d-41b7-9d37-87eb37175b22 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:20.984909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9040cc-22de-4739-b921-619ddbe840dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:20.985800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9040cc-22de-4739-b921-619ddbe840dd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:20.986248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9040cc-22de-4739-b921-619ddbe840dd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:20.986662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9040cc-22de-4739-b921-619ddbe840dd None admin] PATH_INFO: `/v3/projects/2f960a38bb5245d9af199b3ca1376f2b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:20.987942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9040cc-22de-4739-b921-619ddbe840dd None admin] RBAC: Authorizing `identity:get_project(project_id=2f960a38bb5245d9af199b3ca1376f2b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:20.989647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9040cc-22de-4739-b921-619ddbe840dd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:20.991541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 90/184] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:20 2022] GET /identity/v3/projects/2f960a38bb5245d9af199b3ca1376f2b => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:20.992464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9040cc-22de-4739-b921-619ddbe840dd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.001779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249fae0-3b51-4383-b7e1-a97df39e86e4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.038924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f9713-21a9-4f2a-a1c6-d2ea681fcc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.039559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f9713-21a9-4f2a-a1c6-d2ea681fcc9c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.039838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f9713-21a9-4f2a-a1c6-d2ea681fcc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.040105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f9713-21a9-4f2a-a1c6-d2ea681fcc9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.051468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f9713-21a9-4f2a-a1c6-d2ea681fcc9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c48f9713-21a9-4f2a-a1c6-d2ea681fcc9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:21.056082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:21.057193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:21.058216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 95/185] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:09:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:21.063375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec50cb4-ce98-4e30-a77e-a8035b58423b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.063661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec50cb4-ce98-4e30-a77e-a8035b58423b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.063933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec50cb4-ce98-4e30-a77e-a8035b58423b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.081633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dec50cb4-ce98-4e30-a77e-a8035b58423b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:21.082801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dec50cb4-ce98-4e30-a77e-a8035b58423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:21.098123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:21.105644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9040cc-22de-4739-b921-619ddbe840dd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.142178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e593f-611b-448b-98dc-72447933885c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.142893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e593f-611b-448b-98dc-72447933885c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.143174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e593f-611b-448b-98dc-72447933885c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.143453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e593f-611b-448b-98dc-72447933885c None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.144621 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e593f-611b-448b-98dc-72447933885c None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:21.146104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e593f-611b-448b-98dc-72447933885c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:21.147751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 91/186] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:20 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:21.147751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e593f-611b-448b-98dc-72447933885c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.159746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 96/187] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:21.160620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec50cb4-ce98-4e30-a77e-a8035b58423b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.183269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:21.268114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec50cb4-ce98-4e30-a77e-a8035b58423b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.269036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e593f-611b-448b-98dc-72447933885c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.304052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c435e-d26a-423f-8530-8f4f2b9f1898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.304738 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c435e-d26a-423f-8530-8f4f2b9f1898 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.305016 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c435e-d26a-423f-8530-8f4f2b9f1898 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.305286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c435e-d26a-423f-8530-8f4f2b9f1898 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.305835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f3253-c354-4ea2-85ea-2ec4134984b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.306380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c435e-d26a-423f-8530-8f4f2b9f1898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:21.306474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f3253-c354-4ea2-85ea-2ec4134984b3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.306699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f3253-c354-4ea2-85ea-2ec4134984b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.306971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f3253-c354-4ea2-85ea-2ec4134984b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.307702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f3253-c354-4ea2-85ea-2ec4134984b3 None admin] RBAC: Adding query filter params (role.id=a7f0f98b4677401690c354299b049892, scope.project.id=2f960a38bb5245d9af199b3ca1376f2b, user.id=0c47d47164cb4842b0a2341a28ea82f3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:21.307859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c435e-d26a-423f-8530-8f4f2b9f1898 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:21.308003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f3253-c354-4ea2-85ea-2ec4134984b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:21.309573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f3253-c354-4ea2-85ea-2ec4134984b3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:21.310004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 97/188] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:21.310651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c435e-d26a-423f-8530-8f4f2b9f1898 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.320490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 92/189] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:21 2022] GET /identity/v3/role_assignments?user.id=0c47d47164cb4842b0a2341a28ea82f3&scope.project.id=2f960a38bb5245d9af199b3ca1376f2b&role.id=a7f0f98b4677401690c354299b049892 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:21.321279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f3253-c354-4ea2-85ea-2ec4134984b3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.340907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:21.452540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c435e-d26a-423f-8530-8f4f2b9f1898 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.460500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:21.470597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f3253-c354-4ea2-85ea-2ec4134984b3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.496637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab0fe4-24a2-4ea6-ba34-25c47780e698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.498035 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab0fe4-24a2-4ea6-ba34-25c47780e698 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.498340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab0fe4-24a2-4ea6-ba34-25c47780e698 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.498631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab0fe4-24a2-4ea6-ba34-25c47780e698 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.499318 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab0fe4-24a2-4ea6-ba34-25c47780e698 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:21.499779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab0fe4-24a2-4ea6-ba34-25c47780e698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:21.501222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab0fe4-24a2-4ea6-ba34-25c47780e698 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:21.510351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 98/190] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:21.510542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83989752-a7b1-48f1-89e7-8da93a6e2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.511099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83989752-a7b1-48f1-89e7-8da93a6e2f04 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.511234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab0fe4-24a2-4ea6-ba34-25c47780e698 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.512024 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83989752-a7b1-48f1-89e7-8da93a6e2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.512267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83989752-a7b1-48f1-89e7-8da93a6e2f04 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.524099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83989752-a7b1-48f1-89e7-8da93a6e2f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-83989752-a7b1-48f1-89e7-8da93a6e2f04 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:21.530276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:21.531157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:21.531951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 93/191] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:21.537456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83989752-a7b1-48f1-89e7-8da93a6e2f04 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.602970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:21.625069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab0fe4-24a2-4ea6-ba34-25c47780e698 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.639137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83989752-a7b1-48f1-89e7-8da93a6e2f04 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.658954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b26dd-9132-4bd4-b30b-cdc0826626d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.659552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b26dd-9132-4bd4-b30b-cdc0826626d0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.659833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b26dd-9132-4bd4-b30b-cdc0826626d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.660115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b26dd-9132-4bd4-b30b-cdc0826626d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.660871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b26dd-9132-4bd4-b30b-cdc0826626d0 None admin] RBAC: Adding query filter params (role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=c61f47242a0d40f59b342520453f190d, user.id=0c47d47164cb4842b0a2341a28ea82f3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:21.661176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b26dd-9132-4bd4-b30b-cdc0826626d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:21.662638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b26dd-9132-4bd4-b30b-cdc0826626d0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:21.668566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 99/192] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:21 2022] GET /identity/v3/role_assignments?user.id=0c47d47164cb4842b0a2341a28ea82f3&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=048908af57db471987eb7f2ec21e0304 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:21.668566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b26dd-9132-4bd4-b30b-cdc0826626d0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.679939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8af863-06c1-4e4f-84bb-238c2a4c5885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.680618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8af863-06c1-4e4f-84bb-238c2a4c5885 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.680900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8af863-06c1-4e4f-84bb-238c2a4c5885 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.681201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8af863-06c1-4e4f-84bb-238c2a4c5885 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.682542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8af863-06c1-4e4f-84bb-238c2a4c5885 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:21.685288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8af863-06c1-4e4f-84bb-238c2a4c5885 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:21.720421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 94/193] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:09:21 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:09:21.720421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8af863-06c1-4e4f-84bb-238c2a4c5885 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.732265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:21.824625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b26dd-9132-4bd4-b30b-cdc0826626d0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.861261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8af863-06c1-4e4f-84bb-238c2a4c5885 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:21.880158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd18a51-0a82-402a-9036-36a7a03af769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.880829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd18a51-0a82-402a-9036-36a7a03af769 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.881107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd18a51-0a82-402a-9036-36a7a03af769 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.881379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd18a51-0a82-402a-9036-36a7a03af769 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.895998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd18a51-0a82-402a-9036-36a7a03af769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2bd18a51-0a82-402a-9036-36a7a03af769 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:21.903356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:21.904764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 100/194] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:21.906245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd18a51-0a82-402a-9036-36a7a03af769 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:21.911692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d75c31-b97c-43f0-b02c-8d86e1fe78f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:21.912587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d75c31-b97c-43f0-b02c-8d86e1fe78f4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:21.912703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d75c31-b97c-43f0-b02c-8d86e1fe78f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:21.912811 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:21.912990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d75c31-b97c-43f0-b02c-8d86e1fe78f4 None admin] PATH_INFO: `/v3/users/b73b7e1fbf7f450d82c872e822864421` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:21.931939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d75c31-b97c-43f0-b02c-8d86e1fe78f4 None admin] RBAC: Authorizing `identity:get_user(user_id=b73b7e1fbf7f450d82c872e822864421)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:21.934901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d75c31-b97c-43f0-b02c-8d86e1fe78f4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:21.936598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 95/195] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:21 2022] GET /identity/v3/users/b73b7e1fbf7f450d82c872e822864421 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:21.936598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d75c31-b97c-43f0-b02c-8d86e1fe78f4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:22.026175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:22.047038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd18a51-0a82-402a-9036-36a7a03af769 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:22.077651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d75c31-b97c-43f0-b02c-8d86e1fe78f4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:22.094684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9444f-f402-461d-b306-e2c9f087908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:22.095301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9444f-f402-461d-b306-e2c9f087908a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:22.096779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9444f-f402-461d-b306-e2c9f087908a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:22.096779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9444f-f402-461d-b306-e2c9f087908a None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:22.096779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9444f-f402-461d-b306-e2c9f087908a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:22.097045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9444f-f402-461d-b306-e2c9f087908a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:22.098214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9444f-f402-461d-b306-e2c9f087908a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:22.109558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 101/196] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:22.110417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9444f-f402-461d-b306-e2c9f087908a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:22.123734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a3a3-b6b8-4791-b5ba-a9d9b7d980cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:22.124919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a3a3-b6b8-4791-b5ba-a9d9b7d980cd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:22.125479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a3a3-b6b8-4791-b5ba-a9d9b7d980cd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:22.125758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876a3a3-b6b8-4791-b5ba-a9d9b7d980cd None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:22.126445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876a3a3-b6b8-4791-b5ba-a9d9b7d980cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:22.126750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876a3a3-b6b8-4791-b5ba-a9d9b7d980cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:22.128450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876a3a3-b6b8-4791-b5ba-a9d9b7d980cd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:22.144516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 96/197] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:22.145724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a3a3-b6b8-4791-b5ba-a9d9b7d980cd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:22.194781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:22.246927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9444f-f402-461d-b306-e2c9f087908a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:22.284219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876a3a3-b6b8-4791-b5ba-a9d9b7d980cd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:22.290871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c329c9e-beec-4111-856c-40d6aaecd466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:22.291843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c329c9e-beec-4111-856c-40d6aaecd466 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:22.292376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c329c9e-beec-4111-856c-40d6aaecd466 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:22.293048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c329c9e-beec-4111-856c-40d6aaecd466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:22.311818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:22.321003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094433c-6118-4406-a6fc-83bdbffd7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:22.321640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094433c-6118-4406-a6fc-83bdbffd7731 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:22.321912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094433c-6118-4406-a6fc-83bdbffd7731 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:22.322182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094433c-6118-4406-a6fc-83bdbffd7731 None admin] PATH_INFO: `/v3/users/501308170bfe4559b48e10facb420a99` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:22.323314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094433c-6118-4406-a6fc-83bdbffd7731 None admin] RBAC: Authorizing `identity:get_user(user_id=501308170bfe4559b48e10facb420a99)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:22.325423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094433c-6118-4406-a6fc-83bdbffd7731 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:22.331908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 97/198] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:22 2022] GET /identity/v3/users/501308170bfe4559b48e10facb420a99 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:22.332865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094433c-6118-4406-a6fc-83bdbffd7731 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:22.375626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c329c9e-beec-4111-856c-40d6aaecd466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:22.377603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c329c9e-beec-4111-856c-40d6aaecd466 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:22.440840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094433c-6118-4406-a6fc-83bdbffd7731 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:22.456006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:22.488464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1167a2-f600-4c56-915b-12fdf1955d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:22.489151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1167a2-f600-4c56-915b-12fdf1955d5c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:22.489476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1167a2-f600-4c56-915b-12fdf1955d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:22.489753 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1167a2-f600-4c56-915b-12fdf1955d5c None admin] PATH_INFO: `/v3/users/b73b7e1fbf7f450d82c872e822864421` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:22.490896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1167a2-f600-4c56-915b-12fdf1955d5c None admin] RBAC: Authorizing `identity:get_user(user_id=b73b7e1fbf7f450d82c872e822864421)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:22.492886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1167a2-f600-4c56-915b-12fdf1955d5c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:22.494940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 98/199] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:22 2022] GET /identity/v3/users/b73b7e1fbf7f450d82c872e822864421 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:22.497434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1167a2-f600-4c56-915b-12fdf1955d5c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:22.519047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 102/200] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:22.520731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c329c9e-beec-4111-856c-40d6aaecd466 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:22.577713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:22.629440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1167a2-f600-4c56-915b-12fdf1955d5c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:22.661893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c329c9e-beec-4111-856c-40d6aaecd466 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:22.673883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd07fa-f688-4c82-8d32-06699f2daf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:22.674963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd07fa-f688-4c82-8d32-06699f2daf26 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:22.675259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd07fa-f688-4c82-8d32-06699f2daf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:22.675534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd07fa-f688-4c82-8d32-06699f2daf26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:22.719517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3018e5-934f-4882-8432-3ab9f3cba174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:22.720878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3018e5-934f-4882-8432-3ab9f3cba174 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:22.721461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3018e5-934f-4882-8432-3ab9f3cba174 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:22.721925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3018e5-934f-4882-8432-3ab9f3cba174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:22.757276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd07fa-f688-4c82-8d32-06699f2daf26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:22.758764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd07fa-f688-4c82-8d32-06699f2daf26 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:22.764706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:22.818492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3018e5-934f-4882-8432-3ab9f3cba174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:22.821207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3018e5-934f-4882-8432-3ab9f3cba174 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:22.829956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:22.875637 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 99/201] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:22.876455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd07fa-f688-4c82-8d32-06699f2daf26 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:22.954237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 103/202] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:22.954841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3018e5-934f-4882-8432-3ab9f3cba174 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:23.023820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd07fa-f688-4c82-8d32-06699f2daf26 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:23.039586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:23.076774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c34cb7-1ed5-4ecc-9fa3-be716a68c727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:23.077991 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c34cb7-1ed5-4ecc-9fa3-be716a68c727 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:23.078459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c34cb7-1ed5-4ecc-9fa3-be716a68c727 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:23.078746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c34cb7-1ed5-4ecc-9fa3-be716a68c727 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:23.080435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c34cb7-1ed5-4ecc-9fa3-be716a68c727 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:23.083700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c34cb7-1ed5-4ecc-9fa3-be716a68c727 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:23.086731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 100/203] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:22 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:23.087887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c34cb7-1ed5-4ecc-9fa3-be716a68c727 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:23.089790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3018e5-934f-4882-8432-3ab9f3cba174 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:23.128572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f62a58-ef6a-45b6-97d0-dac682c834d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:23.129585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f62a58-ef6a-45b6-97d0-dac682c834d5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:23.130059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f62a58-ef6a-45b6-97d0-dac682c834d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:23.130549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f62a58-ef6a-45b6-97d0-dac682c834d5 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:23.132025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f62a58-ef6a-45b6-97d0-dac682c834d5 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:23.133230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:23.134305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f62a58-ef6a-45b6-97d0-dac682c834d5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:23.136739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 104/204] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:22 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:23.138035 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f62a58-ef6a-45b6-97d0-dac682c834d5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:23.254540 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c34cb7-1ed5-4ecc-9fa3-be716a68c727 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:23.256579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f62a58-ef6a-45b6-97d0-dac682c834d5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:23.296450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:23.303872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27de4e0-fd22-4190-a434-b94c84c27beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:23.304518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27de4e0-fd22-4190-a434-b94c84c27beb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:23.304782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27de4e0-fd22-4190-a434-b94c84c27beb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:23.305052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27de4e0-fd22-4190-a434-b94c84c27beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:23.305787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27de4e0-fd22-4190-a434-b94c84c27beb None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=282dfae79c6e4e20aac9334bc958b580, user.id=b73b7e1fbf7f450d82c872e822864421) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:23.306090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27de4e0-fd22-4190-a434-b94c84c27beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:23.306778 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b7cb2-3979-4918-bbfb-b5de77a94150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:23.307648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27de4e0-fd22-4190-a434-b94c84c27beb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:23.307888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b7cb2-3979-4918-bbfb-b5de77a94150 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:23.308542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b7cb2-3979-4918-bbfb-b5de77a94150 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:23.309011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b7cb2-3979-4918-bbfb-b5de77a94150 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:23.310413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b7cb2-3979-4918-bbfb-b5de77a94150 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:23.312096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b7cb2-3979-4918-bbfb-b5de77a94150 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:23.314008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 101/205] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:23 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:23.315058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b7cb2-3979-4918-bbfb-b5de77a94150 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:23.317455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 105/206] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:23 2022] GET /identity/v3/role_assignments?user.id=b73b7e1fbf7f450d82c872e822864421&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:23.325518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27de4e0-fd22-4190-a434-b94c84c27beb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:23.460086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:23.465881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b7cb2-3979-4918-bbfb-b5de77a94150 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:23.535475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84e327-70e9-49a6-94be-06fde8934624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:23.536098 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84e327-70e9-49a6-94be-06fde8934624 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:23.536379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84e327-70e9-49a6-94be-06fde8934624 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:23.536687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84e327-70e9-49a6-94be-06fde8934624 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:23.537565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84e327-70e9-49a6-94be-06fde8934624 None admin] RBAC: Adding query filter params (role.id=12045ab231c24ebeb9dc0c0b7f6af56a, scope.project.id=c61f47242a0d40f59b342520453f190d, user.id=501308170bfe4559b48e10facb420a99) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:23.537879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84e327-70e9-49a6-94be-06fde8934624 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:23.539411 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84e327-70e9-49a6-94be-06fde8934624 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:23.547071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 102/207] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:23 2022] GET /identity/v3/role_assignments?user.id=501308170bfe4559b48e10facb420a99&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=12045ab231c24ebeb9dc0c0b7f6af56a => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:23.557816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27de4e0-fd22-4190-a434-b94c84c27beb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:23.612304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab9312-082e-4427-8aa7-ad4fc00267de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:23.613308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab9312-082e-4427-8aa7-ad4fc00267de None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:23.613785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab9312-082e-4427-8aa7-ad4fc00267de None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:23.614275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab9312-082e-4427-8aa7-ad4fc00267de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:23.615238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab9312-082e-4427-8aa7-ad4fc00267de None admin] RBAC: Adding query filter params (role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=282dfae79c6e4e20aac9334bc958b580, user.id=b73b7e1fbf7f450d82c872e822864421) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:23.615744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab9312-082e-4427-8aa7-ad4fc00267de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:23.617517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab9312-082e-4427-8aa7-ad4fc00267de None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:23.624158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 106/208] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:23 2022] GET /identity/v3/role_assignments?user.id=b73b7e1fbf7f450d82c872e822864421&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=048908af57db471987eb7f2ec21e0304 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:23.996885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216134-e889-45ce-8119-d5bc7185a5c3 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:23.997226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216134-e889-45ce-8119-d5bc7185a5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:23.997415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216134-e889-45ce-8119-d5bc7185a5c3 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:23.999068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 103/209] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:24.004877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1018a2c-dea8-4513-88c5-d0f2baaa3563 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:24.005809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1018a2c-dea8-4513-88c5-d0f2baaa3563 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:24.006281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1018a2c-dea8-4513-88c5-d0f2baaa3563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:24.033059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a1018a2c-dea8-4513-88c5-d0f2baaa3563 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:24.035324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a1018a2c-dea8-4513-88c5-d0f2baaa3563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:24.106927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:24.119697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 107/210] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:24.147176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f507d8-304e-441e-8a39-56d0ad837836 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:24.147665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f507d8-304e-441e-8a39-56d0ad837836 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:24.147737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f507d8-304e-441e-8a39-56d0ad837836 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:24.149335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 104/211] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:24.156035 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b582b-0722-471b-9f2b-0fb5e8fb775f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:24.156322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b582b-0722-471b-9f2b-0fb5e8fb775f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:24.156643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b582b-0722-471b-9f2b-0fb5e8fb775f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:24.177144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2e8b582b-0722-471b-9f2b-0fb5e8fb775f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:24.178341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e8b582b-0722-471b-9f2b-0fb5e8fb775f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:24.256173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 108/212] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:24.393398 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f507d8-304e-441e-8a39-56d0ad837836 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:24.500534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f507d8-304e-441e-8a39-56d0ad837836 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:24.535493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693cf8e-6d3b-489d-9f6d-f828c3ffdac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:24.536123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693cf8e-6d3b-489d-9f6d-f828c3ffdac3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:24.536420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693cf8e-6d3b-489d-9f6d-f828c3ffdac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:24.536700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693cf8e-6d3b-489d-9f6d-f828c3ffdac3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:24.547559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693cf8e-6d3b-489d-9f6d-f828c3ffdac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2693cf8e-6d3b-489d-9f6d-f828c3ffdac3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:24.552799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:24.553689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:24.554478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 105/213] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:24.562473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b582b-0722-471b-9f2b-0fb5e8fb775f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:24.590130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693cf8e-6d3b-489d-9f6d-f828c3ffdac3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:24.624859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:24.672962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b582b-0722-471b-9f2b-0fb5e8fb775f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:24.700421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:24.713795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693cf8e-6d3b-489d-9f6d-f828c3ffdac3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:24.715278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7146392-4eac-4d71-aa64-42329199483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:24.715894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7146392-4eac-4d71-aa64-42329199483f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:24.716172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7146392-4eac-4d71-aa64-42329199483f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:24.716472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7146392-4eac-4d71-aa64-42329199483f None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:24.717143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7146392-4eac-4d71-aa64-42329199483f None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:24.717448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7146392-4eac-4d71-aa64-42329199483f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:24.718845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7146392-4eac-4d71-aa64-42329199483f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:24.728816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 109/214] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:24.735932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7146392-4eac-4d71-aa64-42329199483f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:24.756237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a65fd0-000f-480c-8930-1996a2b4d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:24.757238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a65fd0-000f-480c-8930-1996a2b4d13b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:24.757521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a65fd0-000f-480c-8930-1996a2b4d13b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:24.757806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a65fd0-000f-480c-8930-1996a2b4d13b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:24.759181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a65fd0-000f-480c-8930-1996a2b4d13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:24.761109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a65fd0-000f-480c-8930-1996a2b4d13b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:24.763118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 106/215] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:24.769881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a65fd0-000f-480c-8930-1996a2b4d13b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:24.864713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7146392-4eac-4d71-aa64-42329199483f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:24.888445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:24.901055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a65fd0-000f-480c-8930-1996a2b4d13b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:24.912574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bad8aa-8de4-4918-a0ad-a9e40b2a3568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:24.913580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bad8aa-8de4-4918-a0ad-a9e40b2a3568 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:24.913869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bad8aa-8de4-4918-a0ad-a9e40b2a3568 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:24.914147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bad8aa-8de4-4918-a0ad-a9e40b2a3568 None admin] PATH_INFO: `/v3/users/d9617ffe7f8847b996e20c13c92c5d25` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:24.931701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bad8aa-8de4-4918-a0ad-a9e40b2a3568 None admin] RBAC: Authorizing `identity:get_user(user_id=d9617ffe7f8847b996e20c13c92c5d25)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:24.933711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bad8aa-8de4-4918-a0ad-a9e40b2a3568 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:24.936424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 110/216] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:24 2022] GET /identity/v3/users/d9617ffe7f8847b996e20c13c92c5d25 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:24.941232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bad8aa-8de4-4918-a0ad-a9e40b2a3568 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:24.948328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a020723-d046-4020-8b38-b511f10d2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:24.949087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a020723-d046-4020-8b38-b511f10d2b8b None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:24.949370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a020723-d046-4020-8b38-b511f10d2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:24.949647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a020723-d046-4020-8b38-b511f10d2b8b None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:24.950325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a020723-d046-4020-8b38-b511f10d2b8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:24.951788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a020723-d046-4020-8b38-b511f10d2b8b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:24.980945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 107/217] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:09:24 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:09:25.000408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:25.071208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bad8aa-8de4-4918-a0ad-a9e40b2a3568 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:25.115517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001b9a-9ba8-4b9e-b60e-33da652f412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:25.116060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001b9a-9ba8-4b9e-b60e-33da652f412f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:25.116345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001b9a-9ba8-4b9e-b60e-33da652f412f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:25.116652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001b9a-9ba8-4b9e-b60e-33da652f412f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:25.192784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f001b9a-9ba8-4b9e-b60e-33da652f412f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:25.194289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f001b9a-9ba8-4b9e-b60e-33da652f412f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:25.238198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:25.303593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 111/218] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:25.310245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a020723-d046-4020-8b38-b511f10d2b8b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:25.429204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a020723-d046-4020-8b38-b511f10d2b8b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:25.474876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2d72e-6bcd-48e3-9350-09c389f7fa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:25.475660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2d72e-6bcd-48e3-9350-09c389f7fa25 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:25.476043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2d72e-6bcd-48e3-9350-09c389f7fa25 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:25.476438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2d72e-6bcd-48e3-9350-09c389f7fa25 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:25.477705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2d72e-6bcd-48e3-9350-09c389f7fa25 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:25.479398 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2d72e-6bcd-48e3-9350-09c389f7fa25 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:25.481800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 108/219] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:25 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:25.487854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001b9a-9ba8-4b9e-b60e-33da652f412f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:25.596591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517f8da-6be8-4c8e-b9f7-4ffdf1b408b8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:25.597039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517f8da-6be8-4c8e-b9f7-4ffdf1b408b8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:25.597412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517f8da-6be8-4c8e-b9f7-4ffdf1b408b8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:25.598868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 109/220] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:09:25.606222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc4a5b-66bb-4fe3-bcc2-72fdbde7b73c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:25.606631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc4a5b-66bb-4fe3-bcc2-72fdbde7b73c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:25.607020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc4a5b-66bb-4fe3-bcc2-72fdbde7b73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:25.612978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001b9a-9ba8-4b9e-b60e-33da652f412f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:25.629362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fefc4a5b-66bb-4fe3-bcc2-72fdbde7b73c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:25.632565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fefc4a5b-66bb-4fe3-bcc2-72fdbde7b73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:25.649859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:25.681173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661fb57-a5c1-4e09-b99b-ba22bc7e23ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:25.683916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661fb57-a5c1-4e09-b99b-ba22bc7e23ef None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:25.684313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661fb57-a5c1-4e09-b99b-ba22bc7e23ef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:25.684715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661fb57-a5c1-4e09-b99b-ba22bc7e23ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:25.685471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661fb57-a5c1-4e09-b99b-ba22bc7e23ef None admin] RBAC: Adding query filter params (role.id=a7f0f98b4677401690c354299b049892, scope.project.id=282dfae79c6e4e20aac9334bc958b580, user.id=d9617ffe7f8847b996e20c13c92c5d25) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:25.685778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661fb57-a5c1-4e09-b99b-ba22bc7e23ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:25.688249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661fb57-a5c1-4e09-b99b-ba22bc7e23ef None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:25.703159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 112/221] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:25 2022] GET /identity/v3/role_assignments?user.id=d9617ffe7f8847b996e20c13c92c5d25&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=a7f0f98b4677401690c354299b049892 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:25.768792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 110/222] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:25.786876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babb58e-16dd-4aac-b468-f9e92a981ede None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:25.787204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babb58e-16dd-4aac-b468-f9e92a981ede None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:25.787429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babb58e-16dd-4aac-b468-f9e92a981ede None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:25.788819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 113/223] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:25.802529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3d0c1-b30b-46d3-bd95-d8c56cd51c68 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:25.802856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3d0c1-b30b-46d3-bd95-d8c56cd51c68 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:25.803086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3d0c1-b30b-46d3-bd95-d8c56cd51c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:25.825690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c2e3d0c1-b30b-46d3-bd95-d8c56cd51c68 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:25.826873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c2e3d0c1-b30b-46d3-bd95-d8c56cd51c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:25.915743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 111/224] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:26.056845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babb58e-16dd-4aac-b468-f9e92a981ede None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:26.061128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:26.177948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babb58e-16dd-4aac-b468-f9e92a981ede None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:26.216870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3d0c1-b30b-46d3-bd95-d8c56cd51c68 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:26.231786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969e5e8-6cb7-4a1b-83a5-3d73815f6f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:26.232446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969e5e8-6cb7-4a1b-83a5-3d73815f6f86 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:26.232737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969e5e8-6cb7-4a1b-83a5-3d73815f6f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:26.233010 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969e5e8-6cb7-4a1b-83a5-3d73815f6f86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:26.256625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969e5e8-6cb7-4a1b-83a5-3d73815f6f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8969e5e8-6cb7-4a1b-83a5-3d73815f6f86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:26.262593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:26.263501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:26.264295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 114/225] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:09:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:26.279301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969e5e8-6cb7-4a1b-83a5-3d73815f6f86 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:26.321066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:26.348001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3d0c1-b30b-46d3-bd95-d8c56cd51c68 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:26.386962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940c83c-a2ca-46f7-93f8-7dae4ba2c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:26.389503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940c83c-a2ca-46f7-93f8-7dae4ba2c4ec None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:26.390725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940c83c-a2ca-46f7-93f8-7dae4ba2c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:26.391864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940c83c-a2ca-46f7-93f8-7dae4ba2c4ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:26.395867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:26.410763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940c83c-a2ca-46f7-93f8-7dae4ba2c4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0940c83c-a2ca-46f7-93f8-7dae4ba2c4ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:26.416033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:26.416931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:26.417881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:26.420973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 112/226] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:26.430796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940c83c-a2ca-46f7-93f8-7dae4ba2c4ec None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:26.476654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969e5e8-6cb7-4a1b-83a5-3d73815f6f86 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:26.539087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992650f3-8701-42c2-9bce-08feebe956cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:26.539730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992650f3-8701-42c2-9bce-08feebe956cd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:26.540013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992650f3-8701-42c2-9bce-08feebe956cd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:26.540290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992650f3-8701-42c2-9bce-08feebe956cd None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:26.541091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992650f3-8701-42c2-9bce-08feebe956cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:26.541323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992650f3-8701-42c2-9bce-08feebe956cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:26.543278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992650f3-8701-42c2-9bce-08feebe956cd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:26.548013 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940c83c-a2ca-46f7-93f8-7dae4ba2c4ec None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:26.566026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 115/227] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:26.567123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992650f3-8701-42c2-9bce-08feebe956cd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:26.589098 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cb1ff-d663-48ed-b9b2-1d06de835829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:26.591605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cb1ff-d663-48ed-b9b2-1d06de835829 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:26.592288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cb1ff-d663-48ed-b9b2-1d06de835829 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:26.593734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cb1ff-d663-48ed-b9b2-1d06de835829 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:26.594608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cb1ff-d663-48ed-b9b2-1d06de835829 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:26.595860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cb1ff-d663-48ed-b9b2-1d06de835829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:26.598258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cb1ff-d663-48ed-b9b2-1d06de835829 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:26.605583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:26.618352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 113/228] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:26.629363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cb1ff-d663-48ed-b9b2-1d06de835829 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:26.768618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992650f3-8701-42c2-9bce-08feebe956cd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:26.816431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:26.816618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cb1ff-d663-48ed-b9b2-1d06de835829 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:26.838584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660f2e4-81de-4589-9956-f6e54be9c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:26.839915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660f2e4-81de-4589-9956-f6e54be9c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:26.840586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660f2e4-81de-4589-9956-f6e54be9c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:26.841061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660f2e4-81de-4589-9956-f6e54be9c0d1 None admin] PATH_INFO: `/v3/users/b73b7e1fbf7f450d82c872e822864421` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:26.842415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660f2e4-81de-4589-9956-f6e54be9c0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=b73b7e1fbf7f450d82c872e822864421)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:26.844081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660f2e4-81de-4589-9956-f6e54be9c0d1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:26.851518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 116/229] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:26 2022] GET /identity/v3/users/b73b7e1fbf7f450d82c872e822864421 => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:26.857348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660f2e4-81de-4589-9956-f6e54be9c0d1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:26.859653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4a505-7487-44ad-b5d4-027a472e2a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:26.860283 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4a505-7487-44ad-b5d4-027a472e2a96 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:26.860613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4a505-7487-44ad-b5d4-027a472e2a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:26.860948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4a505-7487-44ad-b5d4-027a472e2a96 None admin] PATH_INFO: `/v3/users/b73b7e1fbf7f450d82c872e822864421` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:26.862081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4a505-7487-44ad-b5d4-027a472e2a96 None admin] RBAC: Authorizing `identity:get_user(user_id=b73b7e1fbf7f450d82c872e822864421)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:26.863883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4a505-7487-44ad-b5d4-027a472e2a96 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:26.866802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 114/230] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:26 2022] GET /identity/v3/users/b73b7e1fbf7f450d82c872e822864421 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:26.876413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4a505-7487-44ad-b5d4-027a472e2a96 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:27.000935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660f2e4-81de-4589-9956-f6e54be9c0d1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:27.043522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:27.084785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4a505-7487-44ad-b5d4-027a472e2a96 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:27.085849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb078e-8d7b-41d5-a3df-a61b48423d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:27.086503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb078e-8d7b-41d5-a3df-a61b48423d3d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:27.086786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb078e-8d7b-41d5-a3df-a61b48423d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:27.087059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb078e-8d7b-41d5-a3df-a61b48423d3d None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:27.093192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb078e-8d7b-41d5-a3df-a61b48423d3d None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:27.094638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb078e-8d7b-41d5-a3df-a61b48423d3d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:27.096698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 117/231] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:26 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:27.114996 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb078e-8d7b-41d5-a3df-a61b48423d3d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:27.134050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e2e60-0ffd-471c-b029-8f985ed81894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:27.134742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e2e60-0ffd-471c-b029-8f985ed81894 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:27.135027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e2e60-0ffd-471c-b029-8f985ed81894 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:27.135300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e2e60-0ffd-471c-b029-8f985ed81894 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:27.137856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e2e60-0ffd-471c-b029-8f985ed81894 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:27.140806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e2e60-0ffd-471c-b029-8f985ed81894 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:27.142937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 115/232] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:26 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:27.149186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e2e60-0ffd-471c-b029-8f985ed81894 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:27.223154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:27.285973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e2e60-0ffd-471c-b029-8f985ed81894 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:27.295252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb078e-8d7b-41d5-a3df-a61b48423d3d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:27.327471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb27c26-0dd4-4b93-81e2-a66719e3aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:27.329084 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb27c26-0dd4-4b93-81e2-a66719e3aa1e None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:27.329830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb27c26-0dd4-4b93-81e2-a66719e3aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:27.330310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb27c26-0dd4-4b93-81e2-a66719e3aa1e None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580/users/b73b7e1fbf7f450d82c872e822864421/roles/048908af57db471987eb7f2ec21e0304` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:27.333141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb27c26-0dd4-4b93-81e2-a66719e3aa1e None admin] RBAC: Authorizing `identity:create_grant(project_id=282dfae79c6e4e20aac9334bc958b580, user_id=b73b7e1fbf7f450d82c872e822864421, role_id=048908af57db471987eb7f2ec21e0304)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:27.335391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb27c26-0dd4-4b93-81e2-a66719e3aa1e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:27.338051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:27.360434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 116/233] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:09:27 2022] PUT /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580/users/b73b7e1fbf7f450d82c872e822864421/roles/048908af57db471987eb7f2ec21e0304 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:09:27.368272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd7cd2-ce10-40ee-a4f6-6f191d2716ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:27.369415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd7cd2-ce10-40ee-a4f6-6f191d2716ce None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:27.369902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd7cd2-ce10-40ee-a4f6-6f191d2716ce None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:27.370408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd7cd2-ce10-40ee-a4f6-6f191d2716ce None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580/users/b73b7e1fbf7f450d82c872e822864421/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:27.373332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd7cd2-ce10-40ee-a4f6-6f191d2716ce None admin] RBAC: Authorizing `identity:create_grant(project_id=282dfae79c6e4e20aac9334bc958b580, user_id=b73b7e1fbf7f450d82c872e822864421, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:27.375135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd7cd2-ce10-40ee-a4f6-6f191d2716ce None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:27.392860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 118/234] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:09:27 2022] PUT /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580/users/b73b7e1fbf7f450d82c872e822864421/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:09:27.650984 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909cd25-1d2f-4224-bc5e-f3a1201c2d7b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:27.651341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909cd25-1d2f-4224-bc5e-f3a1201c2d7b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:27.651443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909cd25-1d2f-4224-bc5e-f3a1201c2d7b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:27.653269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 117/235] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:27.658616 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9569315-5483-4cef-8c09-c3d97c097673 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:27.658913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9569315-5483-4cef-8c09-c3d97c097673 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:27.659193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9569315-5483-4cef-8c09-c3d97c097673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:27.688099 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f9569315-5483-4cef-8c09-c3d97c097673 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:27.689667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9569315-5483-4cef-8c09-c3d97c097673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:27.708539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750857d-0953-42e9-acc7-f8431ee37cac None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:27.708838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750857d-0953-42e9-acc7-f8431ee37cac None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:27.709106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750857d-0953-42e9-acc7-f8431ee37cac None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:27.710702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 118/236] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:27.716191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e99371-7c6a-4906-8431-d9a3cad35d3c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:27.716514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e99371-7c6a-4906-8431-d9a3cad35d3c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:27.716794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e99371-7c6a-4906-8431-d9a3cad35d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:27.748817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b3e99371-7c6a-4906-8431-d9a3cad35d3c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:27.750015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b3e99371-7c6a-4906-8431-d9a3cad35d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:27.825763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 119/237] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:27.854840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 119/238] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:28.058797 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54d830-0598-4cad-9bd0-df94dbad162f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:28.059483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54d830-0598-4cad-9bd0-df94dbad162f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:28.060026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54d830-0598-4cad-9bd0-df94dbad162f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:28.061940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 120/239] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:28.067064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0d7c7-5656-4fbd-988f-a50468de6c2c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:28.067340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0d7c7-5656-4fbd-988f-a50468de6c2c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:28.067607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d0d7c7-5656-4fbd-988f-a50468de6c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:28.093373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-19d0d7c7-5656-4fbd-988f-a50468de6c2c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:28.094552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-19d0d7c7-5656-4fbd-988f-a50468de6c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:28.180352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 120/240] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:28.227401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54d830-0598-4cad-9bd0-df94dbad162f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:28.234655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0d7c7-5656-4fbd-988f-a50468de6c2c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:28.332667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:28.344967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:28.414424 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d0d7c7-5656-4fbd-988f-a50468de6c2c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:28.439767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54d830-0598-4cad-9bd0-df94dbad162f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:28.486210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1b34a-1b57-43e3-bd19-b7570e74f164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:28.487453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1b34a-1b57-43e3-bd19-b7570e74f164 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:28.488882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1b34a-1b57-43e3-bd19-b7570e74f164 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:28.489381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1b34a-1b57-43e3-bd19-b7570e74f164 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:28.506612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32584f0-8889-4bd8-b834-c4d04d90ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:28.507668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32584f0-8889-4bd8-b834-c4d04d90ce61 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:28.510092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32584f0-8889-4bd8-b834-c4d04d90ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:28.510568 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32584f0-8889-4bd8-b834-c4d04d90ce61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:28.512619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32584f0-8889-4bd8-b834-c4d04d90ce61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:28.513404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1b34a-1b57-43e3-bd19-b7570e74f164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:28.518692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32584f0-8889-4bd8-b834-c4d04d90ce61 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:28.520868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 121/241] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4ed1b34a-1b57-43e3-bd19-b7570e74f164 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:28.521511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:28.522408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:28.523207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 121/242] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:28.529192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32584f0-8889-4bd8-b834-c4d04d90ce61 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:28.529320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1b34a-1b57-43e3-bd19-b7570e74f164 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:28.638982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1b34a-1b57-43e3-bd19-b7570e74f164 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:28.651376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32584f0-8889-4bd8-b834-c4d04d90ce61 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:28.658672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:28.677173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:28.690517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b605f-b131-46b7-86b3-f8513bdb1b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:28.691300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b605f-b131-46b7-86b3-f8513bdb1b8b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:28.691697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b605f-b131-46b7-86b3-f8513bdb1b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:28.692068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b605f-b131-46b7-86b3-f8513bdb1b8b None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:28.692877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b605f-b131-46b7-86b3-f8513bdb1b8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:28.693278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b605f-b131-46b7-86b3-f8513bdb1b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:28.694769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b605f-b131-46b7-86b3-f8513bdb1b8b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:28.695580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634845d9-2d0b-4509-a3d0-06bff7c45726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:28.696376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634845d9-2d0b-4509-a3d0-06bff7c45726 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:28.696816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634845d9-2d0b-4509-a3d0-06bff7c45726 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:28.697222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634845d9-2d0b-4509-a3d0-06bff7c45726 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:28.705021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 122/243] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:28.706199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b605f-b131-46b7-86b3-f8513bdb1b8b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:28.708662 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634845d9-2d0b-4509-a3d0-06bff7c45726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-634845d9-2d0b-4509-a3d0-06bff7c45726 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:28.713863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:28.715642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:28.716781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:28.719498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 122/244] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:28.720607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634845d9-2d0b-4509-a3d0-06bff7c45726 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:28.817873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b605f-b131-46b7-86b3-f8513bdb1b8b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:28.836744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634845d9-2d0b-4509-a3d0-06bff7c45726 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:28.854711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ad9dd-86bf-430e-90bc-026450623513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:28.855478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ad9dd-86bf-430e-90bc-026450623513 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:28.855761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ad9dd-86bf-430e-90bc-026450623513 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:28.856032 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ad9dd-86bf-430e-90bc-026450623513 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:28.856718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ad9dd-86bf-430e-90bc-026450623513 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:28.858195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ad9dd-86bf-430e-90bc-026450623513 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:28.875972 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a99c2-b998-4545-86fe-eefb58807b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:28.876663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a99c2-b998-4545-86fe-eefb58807b4f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:28.876985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a99c2-b998-4545-86fe-eefb58807b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:28.877274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a99c2-b998-4545-86fe-eefb58807b4f None admin] PATH_INFO: `/v3/users/8b769d94e4b24305b0467c9a4d19624c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:28.881454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 123/245] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:09:28 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:09:28.882290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ad9dd-86bf-430e-90bc-026450623513 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:28.894317 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a99c2-b998-4545-86fe-eefb58807b4f None admin] RBAC: Authorizing `identity:get_user(user_id=8b769d94e4b24305b0467c9a4d19624c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:28.895875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a99c2-b998-4545-86fe-eefb58807b4f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:28.897924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 123/246] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:28 2022] GET /identity/v3/users/8b769d94e4b24305b0467c9a4d19624c => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:28.904903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a99c2-b998-4545-86fe-eefb58807b4f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:28.925436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:28.957267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:29.029778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a99c2-b998-4545-86fe-eefb58807b4f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:29.032240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ad9dd-86bf-430e-90bc-026450623513 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:29.071867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8b45b-66ee-4af2-b5ee-63f1ed6dd4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:29.072896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8b45b-66ee-4af2-b5ee-63f1ed6dd4bf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.073395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8b45b-66ee-4af2-b5ee-63f1ed6dd4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.073916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8b45b-66ee-4af2-b5ee-63f1ed6dd4bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.089757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a439923-7720-4071-9c93-e2d7563f003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:29.090842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a439923-7720-4071-9c93-e2d7563f003b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.091319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a439923-7720-4071-9c93-e2d7563f003b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.091808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a439923-7720-4071-9c93-e2d7563f003b None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.092808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a439923-7720-4071-9c93-e2d7563f003b None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:29.093963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a439923-7720-4071-9c93-e2d7563f003b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:29.096726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a439923-7720-4071-9c93-e2d7563f003b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:29.114283 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 124/247] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:29.120063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a439923-7720-4071-9c93-e2d7563f003b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:29.191238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8b45b-66ee-4af2-b5ee-63f1ed6dd4bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:29.193122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8b45b-66ee-4af2-b5ee-63f1ed6dd4bf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:29.234592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:29.252762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:29.259827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a439923-7720-4071-9c93-e2d7563f003b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:29.304416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45f1e-2dbc-4ce1-ac03-f1897e6ca73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:29.304416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45f1e-2dbc-4ce1-ac03-f1897e6ca73e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.304416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45f1e-2dbc-4ce1-ac03-f1897e6ca73e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.304416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45f1e-2dbc-4ce1-ac03-f1897e6ca73e None admin] PATH_INFO: `/v3/users/d9617ffe7f8847b996e20c13c92c5d25` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.304832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c45f1e-2dbc-4ce1-ac03-f1897e6ca73e None admin] RBAC: Authorizing `identity:get_user(user_id=d9617ffe7f8847b996e20c13c92c5d25)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:29.305165 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 124/248] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:09:29.306280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c45f1e-2dbc-4ce1-ac03-f1897e6ca73e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:29.307701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 125/249] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:29 2022] GET /identity/v3/users/d9617ffe7f8847b996e20c13c92c5d25 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:29.308108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8b45b-66ee-4af2-b5ee-63f1ed6dd4bf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:29.313458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45f1e-2dbc-4ce1-ac03-f1897e6ca73e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:29.418769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8b45b-66ee-4af2-b5ee-63f1ed6dd4bf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:29.429849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45f1e-2dbc-4ce1-ac03-f1897e6ca73e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:29.455936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33021d9-f554-4ad3-b021-e1c00fa658ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:29.456926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33021d9-f554-4ad3-b021-e1c00fa658ae None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.457408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33021d9-f554-4ad3-b021-e1c00fa658ae None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.457900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33021d9-f554-4ad3-b021-e1c00fa658ae None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.459251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33021d9-f554-4ad3-b021-e1c00fa658ae None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:29.461169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33021d9-f554-4ad3-b021-e1c00fa658ae None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:29.467251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 125/250] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:29 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:29.470291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33021d9-f554-4ad3-b021-e1c00fa658ae None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:29.471691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18172d40-195d-4091-8e86-0f68fc1eeb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:29.472323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18172d40-195d-4091-8e86-0f68fc1eeb57 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.472648 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18172d40-195d-4091-8e86-0f68fc1eeb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.472959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18172d40-195d-4091-8e86-0f68fc1eeb57 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.474198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18172d40-195d-4091-8e86-0f68fc1eeb57 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:29.476028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18172d40-195d-4091-8e86-0f68fc1eeb57 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:29.478459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 126/251] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:29 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:29.484011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18172d40-195d-4091-8e86-0f68fc1eeb57 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:29.485049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:29.518851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:29.582728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33021d9-f554-4ad3-b021-e1c00fa658ae None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:29.599000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18172d40-195d-4091-8e86-0f68fc1eeb57 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:29.621471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e20e7-1b29-4997-89c8-4af807acdd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:29.622084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e20e7-1b29-4997-89c8-4af807acdd6a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.622379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e20e7-1b29-4997-89c8-4af807acdd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.622696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e20e7-1b29-4997-89c8-4af807acdd6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.623440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e20e7-1b29-4997-89c8-4af807acdd6a None admin] RBAC: Adding query filter params (role.id=12045ab231c24ebeb9dc0c0b7f6af56a, scope.project.id=282dfae79c6e4e20aac9334bc958b580, user.id=8b769d94e4b24305b0467c9a4d19624c) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:29.624114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e20e7-1b29-4997-89c8-4af807acdd6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:29.625964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e20e7-1b29-4997-89c8-4af807acdd6a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:29.636743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 126/252] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:29 2022] GET /identity/v3/role_assignments?user.id=8b769d94e4b24305b0467c9a4d19624c&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=12045ab231c24ebeb9dc0c0b7f6af56a => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:29.653351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa516a53-e126-4f8e-ad67-a2a5e9b6b185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:29.654056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa516a53-e126-4f8e-ad67-a2a5e9b6b185 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.654339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa516a53-e126-4f8e-ad67-a2a5e9b6b185 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.654616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa516a53-e126-4f8e-ad67-a2a5e9b6b185 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580/users/d9617ffe7f8847b996e20c13c92c5d25/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.656925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa516a53-e126-4f8e-ad67-a2a5e9b6b185 None admin] RBAC: Authorizing `identity:create_grant(project_id=282dfae79c6e4e20aac9334bc958b580, user_id=d9617ffe7f8847b996e20c13c92c5d25, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:29.658452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa516a53-e126-4f8e-ad67-a2a5e9b6b185 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:29.688285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 127/253] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:09:29 2022] PUT /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580/users/d9617ffe7f8847b996e20c13c92c5d25/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:09:29.709353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1df81-203d-40c0-97f9-6ea903aecf86 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.709641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1df81-203d-40c0-97f9-6ea903aecf86 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.709890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1df81-203d-40c0-97f9-6ea903aecf86 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.711827 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 127/254] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:29.718317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f7e05-142b-4885-80a8-d486df9186dc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.718643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f7e05-142b-4885-80a8-d486df9186dc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.718945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f7e05-142b-4885-80a8-d486df9186dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.743317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8a1f7e05-142b-4885-80a8-d486df9186dc None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:29.744574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8a1f7e05-142b-4885-80a8-d486df9186dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:29.829908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf8b21-e094-4e30-ab77-4125acef7dd6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.830190 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf8b21-e094-4e30-ab77-4125acef7dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.830592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf8b21-e094-4e30-ab77-4125acef7dd6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.832305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 128/255] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:09:29.836233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 128/256] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:29.838543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16419a36-d222-4ebb-93ed-cb2b86b18f3c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:29.838890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16419a36-d222-4ebb-93ed-cb2b86b18f3c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:29.839163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16419a36-d222-4ebb-93ed-cb2b86b18f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:29.859343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-16419a36-d222-4ebb-93ed-cb2b86b18f3c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:29.860609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-16419a36-d222-4ebb-93ed-cb2b86b18f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:29.934819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:29.939827 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 129/257] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:30.153810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f7e05-142b-4885-80a8-d486df9186dc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:30.178436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:30.272000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f7e05-142b-4885-80a8-d486df9186dc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:30.310127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36468d6c-d154-434d-ba23-9ca8c12891e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:30.310771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36468d6c-d154-434d-ba23-9ca8c12891e8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:30.311054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36468d6c-d154-434d-ba23-9ca8c12891e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:30.311329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36468d6c-d154-434d-ba23-9ca8c12891e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:30.322236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36468d6c-d154-434d-ba23-9ca8c12891e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-36468d6c-d154-434d-ba23-9ca8c12891e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:30.327049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:30.327915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:30.328730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 129/258] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:09:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:30.332100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16419a36-d222-4ebb-93ed-cb2b86b18f3c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:30.335317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36468d6c-d154-434d-ba23-9ca8c12891e8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:30.427992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:30.452190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36468d6c-d154-434d-ba23-9ca8c12891e8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:30.470814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16419a36-d222-4ebb-93ed-cb2b86b18f3c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:30.491826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9990a-0b78-4f3d-a33e-1cb4cefc8bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:30.492760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9990a-0b78-4f3d-a33e-1cb4cefc8bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:30.493187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9990a-0b78-4f3d-a33e-1cb4cefc8bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:30.493634 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9990a-0b78-4f3d-a33e-1cb4cefc8bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:30.494452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9990a-0b78-4f3d-a33e-1cb4cefc8bc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:30.495001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9990a-0b78-4f3d-a33e-1cb4cefc8bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:30.497293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9990a-0b78-4f3d-a33e-1cb4cefc8bc2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:30.506764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 130/259] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:30.507624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ac468-879d-41d3-9d85-ff0308bf24be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:30.508383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ac468-879d-41d3-9d85-ff0308bf24be None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:30.508710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ac468-879d-41d3-9d85-ff0308bf24be None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:30.508983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ac468-879d-41d3-9d85-ff0308bf24be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:30.512164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9990a-0b78-4f3d-a33e-1cb4cefc8bc2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:30.520436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60ac468-879d-41d3-9d85-ff0308bf24be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f60ac468-879d-41d3-9d85-ff0308bf24be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:30.525189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:30.526365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:30.527309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 130/260] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:30.536197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ac468-879d-41d3-9d85-ff0308bf24be None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:30.606839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:30.650179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ac468-879d-41d3-9d85-ff0308bf24be None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:30.651582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9990a-0b78-4f3d-a33e-1cb4cefc8bc2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:30.696471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2f02b-2e2b-4c1d-aeee-353711a1760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:30.697091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2f02b-2e2b-4c1d-aeee-353711a1760f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:30.697426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2f02b-2e2b-4c1d-aeee-353711a1760f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:30.697694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2f02b-2e2b-4c1d-aeee-353711a1760f None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:30.698421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2f02b-2e2b-4c1d-aeee-353711a1760f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:30.698791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2f02b-2e2b-4c1d-aeee-353711a1760f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:30.701341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2f02b-2e2b-4c1d-aeee-353711a1760f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:30.716727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 131/261] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:30.720669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a932f8c-a51d-4e90-ad5a-bcc6c1292621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:30.721397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a932f8c-a51d-4e90-ad5a-bcc6c1292621 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:30.721683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a932f8c-a51d-4e90-ad5a-bcc6c1292621 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:30.721961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a932f8c-a51d-4e90-ad5a-bcc6c1292621 None admin] PATH_INFO: `/v3/users/b73b7e1fbf7f450d82c872e822864421` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:30.723330 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2f02b-2e2b-4c1d-aeee-353711a1760f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:30.724299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a932f8c-a51d-4e90-ad5a-bcc6c1292621 None admin] RBAC: Authorizing `identity:get_user(user_id=b73b7e1fbf7f450d82c872e822864421)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:30.725794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a932f8c-a51d-4e90-ad5a-bcc6c1292621 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:30.735186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 131/262] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:30 2022] GET /identity/v3/users/b73b7e1fbf7f450d82c872e822864421 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:30.740269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a932f8c-a51d-4e90-ad5a-bcc6c1292621 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:30.760983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:30.867317 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2f02b-2e2b-4c1d-aeee-353711a1760f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:30.893020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:30.907317 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8d401-839e-4937-a049-8a7c2eb05d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:30.907954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8d401-839e-4937-a049-8a7c2eb05d5e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:30.908233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8d401-839e-4937-a049-8a7c2eb05d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:30.908535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8d401-839e-4937-a049-8a7c2eb05d5e None admin] PATH_INFO: `/v3/users/b73b7e1fbf7f450d82c872e822864421` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:30.909671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8d401-839e-4937-a049-8a7c2eb05d5e None admin] RBAC: Authorizing `identity:get_user(user_id=b73b7e1fbf7f450d82c872e822864421)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:30.911194 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc8d401-839e-4937-a049-8a7c2eb05d5e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:30.912820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 132/263] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:30 2022] GET /identity/v3/users/b73b7e1fbf7f450d82c872e822864421 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:30.920022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8d401-839e-4937-a049-8a7c2eb05d5e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:30.968275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a932f8c-a51d-4e90-ad5a-bcc6c1292621 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:31.040340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8d401-839e-4937-a049-8a7c2eb05d5e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:31.052928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5e355-9b9b-4eb0-86af-d93657b38ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:31.053826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5e355-9b9b-4eb0-86af-d93657b38ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:31.054328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5e355-9b9b-4eb0-86af-d93657b38ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:31.054788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5e355-9b9b-4eb0-86af-d93657b38ed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:31.105508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513daa4-07a1-4311-9637-e4f22e62c49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:31.106128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513daa4-07a1-4311-9637-e4f22e62c49a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:31.106410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513daa4-07a1-4311-9637-e4f22e62c49a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:31.106682 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513daa4-07a1-4311-9637-e4f22e62c49a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:31.204748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:31.224053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513daa4-07a1-4311-9637-e4f22e62c49a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:31.225755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513daa4-07a1-4311-9637-e4f22e62c49a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:31.253086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5e355-9b9b-4eb0-86af-d93657b38ed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:31.255010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad5e355-9b9b-4eb0-86af-d93657b38ed9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:31.282394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:31.482854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 133/264] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:09:31.491135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513daa4-07a1-4311-9637-e4f22e62c49a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:31.504374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 132/265] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:09:31.509944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5e355-9b9b-4eb0-86af-d93657b38ed9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:31.659573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad5e355-9b9b-4eb0-86af-d93657b38ed9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:31.678945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:31.711223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:31.732378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513daa4-07a1-4311-9637-e4f22e62c49a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:31.738129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f353740-fd89-4206-af21-786520243005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:31.738710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f353740-fd89-4206-af21-786520243005 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:31.738992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f353740-fd89-4206-af21-786520243005 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:31.739265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f353740-fd89-4206-af21-786520243005 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:31.740426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f353740-fd89-4206-af21-786520243005 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:31.741917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f353740-fd89-4206-af21-786520243005 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:31.748753 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 133/266] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:31 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:31.753316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f353740-fd89-4206-af21-786520243005 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:31.818162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee0f5b-01a5-4032-8b93-84c343ecc007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:31.818906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0f5b-01a5-4032-8b93-84c343ecc007 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:31.819218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0f5b-01a5-4032-8b93-84c343ecc007 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:31.819540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee0f5b-01a5-4032-8b93-84c343ecc007 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:31.820750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee0f5b-01a5-4032-8b93-84c343ecc007 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:31.822227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee0f5b-01a5-4032-8b93-84c343ecc007 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:31.824513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 134/267] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:31 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:31.830811 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee0f5b-01a5-4032-8b93-84c343ecc007 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:31.938989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f353740-fd89-4206-af21-786520243005 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:32.007505 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11fa12-6862-49fc-b1e8-ab1346e5598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:32.008031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11fa12-6862-49fc-b1e8-ab1346e5598b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.008313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11fa12-6862-49fc-b1e8-ab1346e5598b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.008622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11fa12-6862-49fc-b1e8-ab1346e5598b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.009363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11fa12-6862-49fc-b1e8-ab1346e5598b None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=282dfae79c6e4e20aac9334bc958b580, user.id=b73b7e1fbf7f450d82c872e822864421) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:32.009673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11fa12-6862-49fc-b1e8-ab1346e5598b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:32.011270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11fa12-6862-49fc-b1e8-ab1346e5598b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:32.022337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 134/268] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:31 2022] GET /identity/v3/role_assignments?user.id=b73b7e1fbf7f450d82c872e822864421&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:32.057534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee0f5b-01a5-4032-8b93-84c343ecc007 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:32.120107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a8324-e188-4143-8839-c0acb08d28fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:32.121398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a8324-e188-4143-8839-c0acb08d28fa None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.121675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a8324-e188-4143-8839-c0acb08d28fa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.121951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a8324-e188-4143-8839-c0acb08d28fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.122700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a8324-e188-4143-8839-c0acb08d28fa None admin] RBAC: Adding query filter params (role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=282dfae79c6e4e20aac9334bc958b580, user.id=b73b7e1fbf7f450d82c872e822864421) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:32.123008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a8324-e188-4143-8839-c0acb08d28fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:32.124550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a8324-e188-4143-8839-c0acb08d28fa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:32.131652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 135/269] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:31 2022] GET /identity/v3/role_assignments?user.id=b73b7e1fbf7f450d82c872e822864421&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=048908af57db471987eb7f2ec21e0304 => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:32.362796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6d6ab-e89a-421a-bd87-d26410266925 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.363281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6d6ab-e89a-421a-bd87-d26410266925 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.363656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6d6ab-e89a-421a-bd87-d26410266925 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.365408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 135/270] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:32.371291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378d6ad-04a3-4b57-9e1f-7e07ddd89f75 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.371695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378d6ad-04a3-4b57-9e1f-7e07ddd89f75 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.372042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378d6ad-04a3-4b57-9e1f-7e07ddd89f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.391861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c378d6ad-04a3-4b57-9e1f-7e07ddd89f75 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:32.393799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c378d6ad-04a3-4b57-9e1f-7e07ddd89f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:32.414174 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:32.441556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584bf3-705b-4965-80a7-f394f33bf256 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.441852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584bf3-705b-4965-80a7-f394f33bf256 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.442115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584bf3-705b-4965-80a7-f394f33bf256 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.443723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 136/271] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:32.449760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161256-362a-4be7-9139-c68773c47b4d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.450052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161256-362a-4be7-9139-c68773c47b4d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.450337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161256-362a-4be7-9139-c68773c47b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.468895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 136/272] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:32.472835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-67161256-362a-4be7-9139-c68773c47b4d None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:32.474055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-67161256-362a-4be7-9139-c68773c47b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:32.515151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:32.554013 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 137/273] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:32.767752 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378d6ad-04a3-4b57-9e1f-7e07ddd89f75 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:32.841580 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a9eed-ea10-4c09-b93a-069d3dc2ccb1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.841899 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a9eed-ea10-4c09-b93a-069d3dc2ccb1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.842960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a9eed-ea10-4c09-b93a-069d3dc2ccb1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.844323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 138/274] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:09:32.846896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910250-565c-408c-95f0-55307214b616 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.847647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910250-565c-408c-95f0-55307214b616 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.847924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910250-565c-408c-95f0-55307214b616 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.849195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 139/275] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:32.851337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c3206-e9c9-49ba-8147-5f1f01902c67 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.851615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c3206-e9c9-49ba-8147-5f1f01902c67 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.851885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c3206-e9c9-49ba-8147-5f1f01902c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.873934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-815c3206-e9c9-49ba-8147-5f1f01902c67 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:32.875484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-815c3206-e9c9-49ba-8147-5f1f01902c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:32.881506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378d6ad-04a3-4b57-9e1f-7e07ddd89f75 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:32.915422 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda9a08-1154-431b-be79-4071a77df5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:32.916051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9a08-1154-431b-be79-4071a77df5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.916325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9a08-1154-431b-be79-4071a77df5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.916622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9a08-1154-431b-be79-4071a77df5d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.927164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda9a08-1154-431b-be79-4071a77df5d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-feda9a08-1154-431b-be79-4071a77df5d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:32.931779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:32.932674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:32.933823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 137/276] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:32.934478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa9574-cd32-42a1-81d7-a4e6c37098d6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:32.934798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa9574-cd32-42a1-81d7-a4e6c37098d6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:32.935117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa9574-cd32-42a1-81d7-a4e6c37098d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:32.947610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 140/277] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:32.948499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c3206-e9c9-49ba-8147-5f1f01902c67 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:32.952307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-44aa9574-cd32-42a1-81d7-a4e6c37098d6 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:32.953507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-44aa9574-cd32-42a1-81d7-a4e6c37098d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:32.961863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:33.031621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 138/278] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:33.032532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa9574-cd32-42a1-81d7-a4e6c37098d6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.073021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c3206-e9c9-49ba-8147-5f1f01902c67 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.075264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:33.128099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014460e-012d-4657-88c9-7bfd8a6a068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.128767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014460e-012d-4657-88c9-7bfd8a6a068b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.129052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014460e-012d-4657-88c9-7bfd8a6a068b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.129329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014460e-012d-4657-88c9-7bfd8a6a068b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.145282 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014460e-012d-4657-88c9-7bfd8a6a068b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e014460e-012d-4657-88c9-7bfd8a6a068b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:33.149974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:33.151134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:33.153130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 141/279] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:33.159993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014460e-012d-4657-88c9-7bfd8a6a068b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.176993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa9574-cd32-42a1-81d7-a4e6c37098d6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.213068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c3c4-97b1-4c39-ab18-8f7d59a7afe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.213605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c3c4-97b1-4c39-ab18-8f7d59a7afe4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.213888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c3c4-97b1-4c39-ab18-8f7d59a7afe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.214770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c3c4-97b1-4c39-ab18-8f7d59a7afe4 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.215863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5c3c4-97b1-4c39-ab18-8f7d59a7afe4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:33.216177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5c3c4-97b1-4c39-ab18-8f7d59a7afe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:33.217628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5c3c4-97b1-4c39-ab18-8f7d59a7afe4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:33.227329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 139/280] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:33.228156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c3c4-97b1-4c39-ab18-8f7d59a7afe4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.232826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:33.271602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014460e-012d-4657-88c9-7bfd8a6a068b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.309661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdb524-2421-4fa9-8e79-db0112dffc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.310263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb524-2421-4fa9-8e79-db0112dffc6c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.310541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb524-2421-4fa9-8e79-db0112dffc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.310809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdb524-2421-4fa9-8e79-db0112dffc6c None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.311474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdb524-2421-4fa9-8e79-db0112dffc6c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:33.311790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdb524-2421-4fa9-8e79-db0112dffc6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:33.313202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdb524-2421-4fa9-8e79-db0112dffc6c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:33.321494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 142/281] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:33.325328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdb524-2421-4fa9-8e79-db0112dffc6c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.337414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c3c4-97b1-4c39-ab18-8f7d59a7afe4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.350730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:33.372514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad7cc-691a-448a-8a4f-f83be7e58df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.373146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad7cc-691a-448a-8a4f-f83be7e58df6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.373425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad7cc-691a-448a-8a4f-f83be7e58df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.373699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad7cc-691a-448a-8a4f-f83be7e58df6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.383676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ad7cc-691a-448a-8a4f-f83be7e58df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-920ad7cc-691a-448a-8a4f-f83be7e58df6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:33.388613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:33.389537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:33.390348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 140/282] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:33.391038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad7cc-691a-448a-8a4f-f83be7e58df6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.433970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdb524-2421-4fa9-8e79-db0112dffc6c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.452689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:33.472990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5b38a-a735-4a90-a08f-f6f3070ecd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.473540 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5b38a-a735-4a90-a08f-f6f3070ecd57 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.473817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5b38a-a735-4a90-a08f-f6f3070ecd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.474112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5b38a-a735-4a90-a08f-f6f3070ecd57 None admin] PATH_INFO: `/v3/users/d9617ffe7f8847b996e20c13c92c5d25` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.475224 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5b38a-a735-4a90-a08f-f6f3070ecd57 None admin] RBAC: Authorizing `identity:get_user(user_id=d9617ffe7f8847b996e20c13c92c5d25)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:33.476721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5b38a-a735-4a90-a08f-f6f3070ecd57 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:33.478288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 143/283] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/users/d9617ffe7f8847b996e20c13c92c5d25 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:33.479067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5b38a-a735-4a90-a08f-f6f3070ecd57 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.491838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad7cc-691a-448a-8a4f-f83be7e58df6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.526399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09e2a-cfbb-417e-9fb6-cd3db5691d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.527139 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09e2a-cfbb-417e-9fb6-cd3db5691d72 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.527561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09e2a-cfbb-417e-9fb6-cd3db5691d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.527843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09e2a-cfbb-417e-9fb6-cd3db5691d72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.528999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09e2a-cfbb-417e-9fb6-cd3db5691d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:33.530475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09e2a-cfbb-417e-9fb6-cd3db5691d72 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:33.532043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 141/284] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:33.532831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09e2a-cfbb-417e-9fb6-cd3db5691d72 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.579179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:33.588351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5b38a-a735-4a90-a08f-f6f3070ecd57 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.622834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb6895-38fc-4b50-8b91-dc751b40c416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.623457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb6895-38fc-4b50-8b91-dc751b40c416 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.623741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb6895-38fc-4b50-8b91-dc751b40c416 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.624017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb6895-38fc-4b50-8b91-dc751b40c416 None admin] PATH_INFO: `/v3/users/8b769d94e4b24305b0467c9a4d19624c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.625177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb6895-38fc-4b50-8b91-dc751b40c416 None admin] RBAC: Authorizing `identity:get_user(user_id=8b769d94e4b24305b0467c9a4d19624c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:33.626642 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb6895-38fc-4b50-8b91-dc751b40c416 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:33.628221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 144/285] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/users/8b769d94e4b24305b0467c9a4d19624c => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:33.632830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb6895-38fc-4b50-8b91-dc751b40c416 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.638726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09e2a-cfbb-417e-9fb6-cd3db5691d72 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.671124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:33.676028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed8393-441d-4c90-84df-8fe19df5645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.676683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed8393-441d-4c90-84df-8fe19df5645b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.676958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed8393-441d-4c90-84df-8fe19df5645b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.677225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed8393-441d-4c90-84df-8fe19df5645b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.677889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed8393-441d-4c90-84df-8fe19df5645b None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:33.678189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed8393-441d-4c90-84df-8fe19df5645b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:33.679573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed8393-441d-4c90-84df-8fe19df5645b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:33.688730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 142/286] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:33.689515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed8393-441d-4c90-84df-8fe19df5645b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.733652 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb6895-38fc-4b50-8b91-dc751b40c416 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.765574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c6a14-8ab2-46f6-8aa3-7ddd4ff7d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.766168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c6a14-8ab2-46f6-8aa3-7ddd4ff7d33f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.766445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c6a14-8ab2-46f6-8aa3-7ddd4ff7d33f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.766790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c6a14-8ab2-46f6-8aa3-7ddd4ff7d33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.794031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed8393-441d-4c90-84df-8fe19df5645b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:33.795244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:33.830891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d754df8-0fd4-433e-b26f-8b5663a25bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:33.831497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d754df8-0fd4-433e-b26f-8b5663a25bc8 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:33.831777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d754df8-0fd4-433e-b26f-8b5663a25bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:33.832054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d754df8-0fd4-433e-b26f-8b5663a25bc8 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:33.832730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d754df8-0fd4-433e-b26f-8b5663a25bc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:33.834192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d754df8-0fd4-433e-b26f-8b5663a25bc8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:33.836565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c6a14-8ab2-46f6-8aa3-7ddd4ff7d33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:33.838030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c6a14-8ab2-46f6-8aa3-7ddd4ff7d33f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:33.855516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 143/287] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:09:33 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:09:33.856420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d754df8-0fd4-433e-b26f-8b5663a25bc8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.926624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:33.958995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 145/288] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:33.960858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c6a14-8ab2-46f6-8aa3-7ddd4ff7d33f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:33.991077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d754df8-0fd4-433e-b26f-8b5663a25bc8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:34.031298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:34.038528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9692dd6-2224-413c-9a2d-2168b20bdc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:34.039166 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9692dd6-2224-413c-9a2d-2168b20bdc48 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:34.039439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9692dd6-2224-413c-9a2d-2168b20bdc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:34.039711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9692dd6-2224-413c-9a2d-2168b20bdc48 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:34.041051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9692dd6-2224-413c-9a2d-2168b20bdc48 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:34.042663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9692dd6-2224-413c-9a2d-2168b20bdc48 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:34.048446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 144/289] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:34.048446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9692dd6-2224-413c-9a2d-2168b20bdc48 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:34.082185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c6a14-8ab2-46f6-8aa3-7ddd4ff7d33f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:34.125627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4946f7-0e62-4c02-b232-05741d4d8ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:34.126269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4946f7-0e62-4c02-b232-05741d4d8ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:34.126540 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4946f7-0e62-4c02-b232-05741d4d8ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:34.126817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4946f7-0e62-4c02-b232-05741d4d8ba0 None admin] PATH_INFO: `/v3/users/58dc8e8334a04b4f914dfd7c36bd02fc` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:34.146969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4946f7-0e62-4c02-b232-05741d4d8ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=58dc8e8334a04b4f914dfd7c36bd02fc)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:34.148604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4946f7-0e62-4c02-b232-05741d4d8ba0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:34.150901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 146/290] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:33 2022] GET /identity/v3/users/58dc8e8334a04b4f914dfd7c36bd02fc => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:34.152285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4946f7-0e62-4c02-b232-05741d4d8ba0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:34.179288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9692dd6-2224-413c-9a2d-2168b20bdc48 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:34.203009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:34.231818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a2b11-2585-4e3f-b4bd-91afe431ca75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:34.232934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a2b11-2585-4e3f-b4bd-91afe431ca75 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:34.233437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a2b11-2585-4e3f-b4bd-91afe431ca75 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:34.233958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a2b11-2585-4e3f-b4bd-91afe431ca75 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:34.235285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a2b11-2585-4e3f-b4bd-91afe431ca75 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:34.237231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a2b11-2585-4e3f-b4bd-91afe431ca75 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:34.239186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 145/291] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:34 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:34.244995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a2b11-2585-4e3f-b4bd-91afe431ca75 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:34.278346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4946f7-0e62-4c02-b232-05741d4d8ba0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:34.298404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:34.318426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbf0e6-9032-42c1-8bc5-08713a7d4c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:34.319421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbf0e6-9032-42c1-8bc5-08713a7d4c31 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:34.319957 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbf0e6-9032-42c1-8bc5-08713a7d4c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:34.320449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbf0e6-9032-42c1-8bc5-08713a7d4c31 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580/users/8b769d94e4b24305b0467c9a4d19624c/roles/12045ab231c24ebeb9dc0c0b7f6af56a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:34.322593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbf0e6-9032-42c1-8bc5-08713a7d4c31 None admin] RBAC: Authorizing `identity:create_grant(project_id=282dfae79c6e4e20aac9334bc958b580, user_id=8b769d94e4b24305b0467c9a4d19624c, role_id=12045ab231c24ebeb9dc0c0b7f6af56a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:34.324325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbf0e6-9032-42c1-8bc5-08713a7d4c31 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:34.340518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 147/292] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:09:34 2022] PUT /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580/users/8b769d94e4b24305b0467c9a4d19624c/roles/12045ab231c24ebeb9dc0c0b7f6af56a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:09:34.341968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbf0e6-9032-42c1-8bc5-08713a7d4c31 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:34.398470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a2b11-2585-4e3f-b4bd-91afe431ca75 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:34.466560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbf0e6-9032-42c1-8bc5-08713a7d4c31 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:34.490817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff021cf-1e34-4036-9e1a-c3253e017c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:34.491434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff021cf-1e34-4036-9e1a-c3253e017c06 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:34.491711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff021cf-1e34-4036-9e1a-c3253e017c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:34.491984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff021cf-1e34-4036-9e1a-c3253e017c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:34.492818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff021cf-1e34-4036-9e1a-c3253e017c06 None admin] RBAC: Adding query filter params (role.id=a7f0f98b4677401690c354299b049892, user.id=58dc8e8334a04b4f914dfd7c36bd02fc, scope.system=all) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:34.493051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff021cf-1e34-4036-9e1a-c3253e017c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:34.494540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff021cf-1e34-4036-9e1a-c3253e017c06 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:34.501721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 146/293] 10.222.0.75 () {56 vars in 1308 bytes} [Fri Aug 19 15:09:34 2022] GET /identity/v3/role_assignments?user.id=58dc8e8334a04b4f914dfd7c36bd02fc&scope.system=all&role.id=a7f0f98b4677401690c354299b049892 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:34.509513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621661c-e674-410c-bfc1-585bd4f9f40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:34.510534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621661c-e674-410c-bfc1-585bd4f9f40f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:34.511053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621661c-e674-410c-bfc1-585bd4f9f40f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:34.511558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621661c-e674-410c-bfc1-585bd4f9f40f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:34.512613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621661c-e674-410c-bfc1-585bd4f9f40f None admin] RBAC: Adding query filter params (role.id=a7f0f98b4677401690c354299b049892, scope.project.id=282dfae79c6e4e20aac9334bc958b580, user.id=d9617ffe7f8847b996e20c13c92c5d25) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:34.513141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621661c-e674-410c-bfc1-585bd4f9f40f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:34.514850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621661c-e674-410c-bfc1-585bd4f9f40f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:34.522675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 148/294] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:34 2022] GET /identity/v3/role_assignments?user.id=d9617ffe7f8847b996e20c13c92c5d25&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=a7f0f98b4677401690c354299b049892 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:36.010090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e380e8-b58d-4194-b0a6-a961ff3f5205 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.011001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e380e8-b58d-4194-b0a6-a961ff3f5205 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.011460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e380e8-b58d-4194-b0a6-a961ff3f5205 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.013268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 147/295] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:36.018904 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e48b7-8e30-4654-a575-808852c47cb6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.019734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e48b7-8e30-4654-a575-808852c47cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.020210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e48b7-8e30-4654-a575-808852c47cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.039853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c21e48b7-8e30-4654-a575-808852c47cb6 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:36.041395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c21e48b7-8e30-4654-a575-808852c47cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:36.082345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:36.082532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62585f4e-750a-4b34-a99e-7a733b53eda1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.082683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62585f4e-750a-4b34-a99e-7a733b53eda1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.082997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62585f4e-750a-4b34-a99e-7a733b53eda1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.084303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 148/296] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:36.091329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb012a-7736-4de1-9d1c-73094059a111 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.091629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb012a-7736-4de1-9d1c-73094059a111 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.091881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb012a-7736-4de1-9d1c-73094059a111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.116440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-96fb012a-7736-4de1-9d1c-73094059a111 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:36.116440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-96fb012a-7736-4de1-9d1c-73094059a111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:36.127050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 149/297] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:36.170226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:36.192305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 149/298] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:36.407869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e48b7-8e30-4654-a575-808852c47cb6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:36.455288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e2e3-2bfe-43c3-9900-794e19256b15 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.455707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e2e3-2bfe-43c3-9900-794e19256b15 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.455952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e2e3-2bfe-43c3-9900-794e19256b15 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.457518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 150/299] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:36.462630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b908f3-47db-469b-bd75-c935e0cfe9dc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.462912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b908f3-47db-469b-bd75-c935e0cfe9dc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.463192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b908f3-47db-469b-bd75-c935e0cfe9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.481944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-80b908f3-47db-469b-bd75-c935e0cfe9dc None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:36.483136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-80b908f3-47db-469b-bd75-c935e0cfe9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:36.536266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e48b7-8e30-4654-a575-808852c47cb6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:36.568999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 151/300] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:36.570130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b908f3-47db-469b-bd75-c935e0cfe9dc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:36.579494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa31f6-09c2-475c-b182-06181104339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:36.580462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa31f6-09c2-475c-b182-06181104339b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.581408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa31f6-09c2-475c-b182-06181104339b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.581895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa31f6-09c2-475c-b182-06181104339b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.597710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa31f6-09c2-475c-b182-06181104339b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-7aaa31f6-09c2-475c-b182-06181104339b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:36.604836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:36.605846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:36.606632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:36.609543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 150/301] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:36.610871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe247db-6a56-40e9-ade6-5db2c1061ba8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.611296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe247db-6a56-40e9-ade6-5db2c1061ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.611717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe247db-6a56-40e9-ade6-5db2c1061ba8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.613164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 151/302] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:36.617813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe247db-6a56-40e9-ade6-5db2c1061ba8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:36.657014 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:36.684811 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b908f3-47db-469b-bd75-c935e0cfe9dc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:36.697218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:36.723517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e7f64-8d2a-4cd1-9b11-e6937c4f413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:36.724805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e7f64-8d2a-4cd1-9b11-e6937c4f413b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.725139 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e7f64-8d2a-4cd1-9b11-e6937c4f413b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.725421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e7f64-8d2a-4cd1-9b11-e6937c4f413b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.726581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e7f64-8d2a-4cd1-9b11-e6937c4f413b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:36.728083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e7f64-8d2a-4cd1-9b11-e6937c4f413b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:36.729839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 152/303] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:36.731241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd724af-a888-4769-a87a-8d409b1d2fd2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.731532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd724af-a888-4769-a87a-8d409b1d2fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.731808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd724af-a888-4769-a87a-8d409b1d2fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.742776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe247db-6a56-40e9-ade6-5db2c1061ba8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:36.750325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dcd724af-a888-4769-a87a-8d409b1d2fd2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:36.751512 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dcd724af-a888-4769-a87a-8d409b1d2fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:36.780829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bd2c6-fc2b-4db1-bae4-53e7a9ae3642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:36.781435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bd2c6-fc2b-4db1-bae4-53e7a9ae3642 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.781717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bd2c6-fc2b-4db1-bae4-53e7a9ae3642 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.781990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bd2c6-fc2b-4db1-bae4-53e7a9ae3642 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.782778 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bd2c6-fc2b-4db1-bae4-53e7a9ae3642 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:36.783349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bd2c6-fc2b-4db1-bae4-53e7a9ae3642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:36.784647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bd2c6-fc2b-4db1-bae4-53e7a9ae3642 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:36.797011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 152/304] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:36.798529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bd2c6-fc2b-4db1-bae4-53e7a9ae3642 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:36.821261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 153/305] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:36.821924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd724af-a888-4769-a87a-8d409b1d2fd2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:36.911331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:36.920102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bd2c6-fc2b-4db1-bae4-53e7a9ae3642 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:36.927201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd724af-a888-4769-a87a-8d409b1d2fd2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:36.937471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:36.963048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868c6fb-586e-4929-b542-b07acc5f2c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:36.964002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9863cd-1737-4a98-be42-3d41b6ae4718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:36.964182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868c6fb-586e-4929-b542-b07acc5f2c91 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.964681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868c6fb-586e-4929-b542-b07acc5f2c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.964787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9863cd-1737-4a98-be42-3d41b6ae4718 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:36.965094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9863cd-1737-4a98-be42-3d41b6ae4718 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:36.965193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868c6fb-586e-4929-b542-b07acc5f2c91 None admin] PATH_INFO: `/v3/groups` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.965357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9863cd-1737-4a98-be42-3d41b6ae4718 None admin] PATH_INFO: `/v3/users/9300370a5d0345d09dcde553bb85dab2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:36.966038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868c6fb-586e-4929-b542-b07acc5f2c91 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:36.967895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868c6fb-586e-4929-b542-b07acc5f2c91 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:36.980302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9863cd-1737-4a98-be42-3d41b6ae4718 None admin] RBAC: Authorizing `identity:get_user(user_id=9300370a5d0345d09dcde553bb85dab2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:36.981814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9863cd-1737-4a98-be42-3d41b6ae4718 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:36.983531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 154/306] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:36 2022] GET /identity/v3/users/9300370a5d0345d09dcde553bb85dab2 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:36.984242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9863cd-1737-4a98-be42-3d41b6ae4718 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:37.000784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 153/307] 10.222.0.75 () {60 vars in 1134 bytes} [Fri Aug 19 15:09:36 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:09:37.001676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868c6fb-586e-4929-b542-b07acc5f2c91 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:37.133129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868c6fb-586e-4929-b542-b07acc5f2c91 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:37.134401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9863cd-1737-4a98-be42-3d41b6ae4718 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:37.173782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d8df4-9aa2-4fb3-9060-34db8b7d8599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:37.174384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8df4-9aa2-4fb3-9060-34db8b7d8599 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:37.174661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8df4-9aa2-4fb3-9060-34db8b7d8599 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:37.174932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8df4-9aa2-4fb3-9060-34db8b7d8599 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:37.175664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d8df4-9aa2-4fb3-9060-34db8b7d8599 None admin] RBAC: Adding query filter params (role.id=12045ab231c24ebeb9dc0c0b7f6af56a, user.id=9300370a5d0345d09dcde553bb85dab2, scope.system=all) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:37.176689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d8df4-9aa2-4fb3-9060-34db8b7d8599 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:37.177532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d8df4-9aa2-4fb3-9060-34db8b7d8599 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:37.182346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea59e4-0c87-4c1b-95c2-d27f12b55beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:37.184437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea59e4-0c87-4c1b-95c2-d27f12b55beb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:37.184584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 154/308] 10.222.0.75 () {56 vars in 1308 bytes} [Fri Aug 19 15:09:37 2022] GET /identity/v3/role_assignments?user.id=9300370a5d0345d09dcde553bb85dab2&scope.system=all&role.id=12045ab231c24ebeb9dc0c0b7f6af56a => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:37.184664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea59e4-0c87-4c1b-95c2-d27f12b55beb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:37.184664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea59e4-0c87-4c1b-95c2-d27f12b55beb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:37.184785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d8df4-9aa2-4fb3-9060-34db8b7d8599 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:37.198230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea59e4-0c87-4c1b-95c2-d27f12b55beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1bea59e4-0c87-4c1b-95c2-d27f12b55beb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:37.204116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:37.205108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:37.205907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:37.207793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 155/309] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:37.214832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea59e4-0c87-4c1b-95c2-d27f12b55beb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:37.218000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:37.254595 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:37.319273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d8df4-9aa2-4fb3-9060-34db8b7d8599 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:37.352312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea59e4-0c87-4c1b-95c2-d27f12b55beb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:37.357429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77376139-2ebb-4584-89ec-bde0f99034a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:37.358094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77376139-2ebb-4584-89ec-bde0f99034a9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:37.358373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77376139-2ebb-4584-89ec-bde0f99034a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:37.358644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77376139-2ebb-4584-89ec-bde0f99034a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:37.371434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77376139-2ebb-4584-89ec-bde0f99034a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-77376139-2ebb-4584-89ec-bde0f99034a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:37.378346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:37.379260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:37.380042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:37.381269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 155/310] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:37.390223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77376139-2ebb-4584-89ec-bde0f99034a9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:37.399338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf273cae-9705-42f4-8ef2-b088cb1a7b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:37.399949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf273cae-9705-42f4-8ef2-b088cb1a7b30 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:37.400229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf273cae-9705-42f4-8ef2-b088cb1a7b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:37.400812 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf273cae-9705-42f4-8ef2-b088cb1a7b30 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:37.401633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf273cae-9705-42f4-8ef2-b088cb1a7b30 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:37.401998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf273cae-9705-42f4-8ef2-b088cb1a7b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:37.403412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf273cae-9705-42f4-8ef2-b088cb1a7b30 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:37.417470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 156/311] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:37.418590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf273cae-9705-42f4-8ef2-b088cb1a7b30 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:37.488464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:37.515153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77376139-2ebb-4584-89ec-bde0f99034a9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:37.520343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:37.533505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf273cae-9705-42f4-8ef2-b088cb1a7b30 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:37.554321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6970c-9e22-4ff8-8c08-cc821729b6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:37.554933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6970c-9e22-4ff8-8c08-cc821729b6bb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:37.555213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6970c-9e22-4ff8-8c08-cc821729b6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:37.555486 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6970c-9e22-4ff8-8c08-cc821729b6bb None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:37.556458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6970c-9e22-4ff8-8c08-cc821729b6bb None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:37.556640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6970c-9e22-4ff8-8c08-cc821729b6bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:37.558563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6970c-9e22-4ff8-8c08-cc821729b6bb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:37.567957 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 156/312] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:37.574583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6970c-9e22-4ff8-8c08-cc821729b6bb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:37.576306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb539143-18ed-4559-bef5-2ea8612b726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:37.577202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb539143-18ed-4559-bef5-2ea8612b726d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:37.577681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb539143-18ed-4559-bef5-2ea8612b726d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:37.578173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb539143-18ed-4559-bef5-2ea8612b726d None admin] PATH_INFO: `/v3/users/8b769d94e4b24305b0467c9a4d19624c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:37.579694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb539143-18ed-4559-bef5-2ea8612b726d None admin] RBAC: Authorizing `identity:get_user(user_id=8b769d94e4b24305b0467c9a4d19624c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:37.581401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb539143-18ed-4559-bef5-2ea8612b726d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:37.583222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 157/313] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:37 2022] GET /identity/v3/users/8b769d94e4b24305b0467c9a4d19624c => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:37.588832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb539143-18ed-4559-bef5-2ea8612b726d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:37.684418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6970c-9e22-4ff8-8c08-cc821729b6bb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:37.704186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb539143-18ed-4559-bef5-2ea8612b726d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:37.721743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0fe7f1-b679-442a-9b28-3b8bc56e6344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:37.722474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fe7f1-b679-442a-9b28-3b8bc56e6344 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:37.722807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fe7f1-b679-442a-9b28-3b8bc56e6344 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:37.723112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fe7f1-b679-442a-9b28-3b8bc56e6344 None admin] PATH_INFO: `/v3/users/58dc8e8334a04b4f914dfd7c36bd02fc` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:37.724318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0fe7f1-b679-442a-9b28-3b8bc56e6344 None admin] RBAC: Authorizing `identity:get_user(user_id=58dc8e8334a04b4f914dfd7c36bd02fc)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:37.725871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0fe7f1-b679-442a-9b28-3b8bc56e6344 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:37.727778 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 157/314] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:37 2022] GET /identity/v3/users/58dc8e8334a04b4f914dfd7c36bd02fc => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:37.734404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0fe7f1-b679-442a-9b28-3b8bc56e6344 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:37.741683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71998122-006a-410b-9ccd-fc1fdae6a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:37.742234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71998122-006a-410b-9ccd-fc1fdae6a507 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:37.742515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71998122-006a-410b-9ccd-fc1fdae6a507 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:37.742795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71998122-006a-410b-9ccd-fc1fdae6a507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:37.747207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:37.771468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:37.817548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71998122-006a-410b-9ccd-fc1fdae6a507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:37.819040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71998122-006a-410b-9ccd-fc1fdae6a507 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:37.846539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0fe7f1-b679-442a-9b28-3b8bc56e6344 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:37.885289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb9463-d5d3-4fb6-b0a4-c0dbe3577f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:37.886383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb9463-d5d3-4fb6-b0a4-c0dbe3577f93 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:37.886743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb9463-d5d3-4fb6-b0a4-c0dbe3577f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:37.887164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb9463-d5d3-4fb6-b0a4-c0dbe3577f93 None admin] PATH_INFO: `/v3/system/users/58dc8e8334a04b4f914dfd7c36bd02fc/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:37.888700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb9463-d5d3-4fb6-b0a4-c0dbe3577f93 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=58dc8e8334a04b4f914dfd7c36bd02fc, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:37.890475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb9463-d5d3-4fb6-b0a4-c0dbe3577f93 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:37.913017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 158/315] 10.222.0.75 () {58 vars in 1334 bytes} [Fri Aug 19 15:09:37 2022] PUT /identity/v3/system/users/58dc8e8334a04b4f914dfd7c36bd02fc/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:09:37.934428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 158/316] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:09:37.940443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb9463-d5d3-4fb6-b0a4-c0dbe3577f93 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:38.020169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:38.054551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb9463-d5d3-4fb6-b0a4-c0dbe3577f93 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:38.091231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f82535-ab2d-4fcd-88c9-166ca1406521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:38.091865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f82535-ab2d-4fcd-88c9-166ca1406521 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:38.092143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f82535-ab2d-4fcd-88c9-166ca1406521 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:38.092428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f82535-ab2d-4fcd-88c9-166ca1406521 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:38.093586 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f82535-ab2d-4fcd-88c9-166ca1406521 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:38.095051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f82535-ab2d-4fcd-88c9-166ca1406521 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:38.097069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 159/317] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:37 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:38.102667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71998122-006a-410b-9ccd-fc1fdae6a507 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:38.169037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:38.220938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71998122-006a-410b-9ccd-fc1fdae6a507 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:38.256999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d3005-9d43-451b-96cd-28735aad453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:38.258003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d3005-9d43-451b-96cd-28735aad453b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:38.258474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d3005-9d43-451b-96cd-28735aad453b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:38.258935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d3005-9d43-451b-96cd-28735aad453b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:38.259856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d3005-9d43-451b-96cd-28735aad453b None admin] RBAC: Adding query filter params (role.id=12045ab231c24ebeb9dc0c0b7f6af56a, scope.project.id=282dfae79c6e4e20aac9334bc958b580, user.id=8b769d94e4b24305b0467c9a4d19624c) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:38.260366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d3005-9d43-451b-96cd-28735aad453b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:38.262057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d3005-9d43-451b-96cd-28735aad453b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:38.268818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 159/318] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:09:38 2022] GET /identity/v3/role_assignments?user.id=8b769d94e4b24305b0467c9a4d19624c&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=12045ab231c24ebeb9dc0c0b7f6af56a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:39.059710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247dbdb2-629f-4de5-ab71-398d677d065c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.060230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247dbdb2-629f-4de5-ab71-398d677d065c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.060230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247dbdb2-629f-4de5-ab71-398d677d065c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.061637 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 160/319] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:39.066813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdad71-5b93-4674-971c-5263f7a0aa60 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.067100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdad71-5b93-4674-971c-5263f7a0aa60 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.067367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdad71-5b93-4674-971c-5263f7a0aa60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.085738 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dabdad71-5b93-4674-971c-5263f7a0aa60 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:39.086911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dabdad71-5b93-4674-971c-5263f7a0aa60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:39.156438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 160/320] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:39.208713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9b568-bcba-4612-a9f5-e8390a5b6841 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.209000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9b568-bcba-4612-a9f5-e8390a5b6841 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.209275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9b568-bcba-4612-a9f5-e8390a5b6841 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.210781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 161/321] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:39.216164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d79322-bf44-4272-88a9-c214aa333adf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.216632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d79322-bf44-4272-88a9-c214aa333adf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.216786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d79322-bf44-4272-88a9-c214aa333adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.234588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-06d79322-bf44-4272-88a9-c214aa333adf None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:39.235835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-06d79322-bf44-4272-88a9-c214aa333adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:39.283883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:39.306583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 161/322] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:39.429398 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9b568-bcba-4612-a9f5-e8390a5b6841 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:39.530832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9b568-bcba-4612-a9f5-e8390a5b6841 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:39.564686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650890fd-662b-4eb4-9401-1e491347fcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:39.565310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650890fd-662b-4eb4-9401-1e491347fcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.565590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650890fd-662b-4eb4-9401-1e491347fcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.565863 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650890fd-662b-4eb4-9401-1e491347fcd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.566961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650890fd-662b-4eb4-9401-1e491347fcd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:39.568427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650890fd-662b-4eb4-9401-1e491347fcd3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:39.570003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 162/323] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:09:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:39.575684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d79322-bf44-4272-88a9-c214aa333adf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:39.589906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650890fd-662b-4eb4-9401-1e491347fcd3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:39.601123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:39.687690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d79322-bf44-4272-88a9-c214aa333adf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:39.704522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650890fd-662b-4eb4-9401-1e491347fcd3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:39.724147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d197962-d304-4137-a814-808892985a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:39.724797 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d197962-d304-4137-a814-808892985a7a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.725082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d197962-d304-4137-a814-808892985a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.725360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d197962-d304-4137-a814-808892985a7a None admin] PATH_INFO: `/v3/groups` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.725995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d197962-d304-4137-a814-808892985a7a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:39.727459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d197962-d304-4137-a814-808892985a7a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:39.741460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a663a19-7ec6-47ef-bfc7-db9a49d72b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:39.742080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a663a19-7ec6-47ef-bfc7-db9a49d72b8e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.742358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a663a19-7ec6-47ef-bfc7-db9a49d72b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.742624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a663a19-7ec6-47ef-bfc7-db9a49d72b8e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.745454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 162/324] 10.222.0.75 () {60 vars in 1134 bytes} [Fri Aug 19 15:09:39 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:09:39.764814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a663a19-7ec6-47ef-bfc7-db9a49d72b8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9a663a19-7ec6-47ef-bfc7-db9a49d72b8e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:39.770326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:39.771286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:39.772096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:39.773354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 163/325] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:39.778513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d197962-d304-4137-a814-808892985a7a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:39.828687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:39.858849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1e7ab-4e1e-4209-8e83-438d934ab799 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.859198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1e7ab-4e1e-4209-8e83-438d934ab799 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.859394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1e7ab-4e1e-4209-8e83-438d934ab799 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.860976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 164/326] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:09:39.866214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a44fa-57f3-4484-acec-b19e25151737 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.866496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a44fa-57f3-4484-acec-b19e25151737 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.866767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a44fa-57f3-4484-acec-b19e25151737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.888308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4c7a44fa-57f3-4484-acec-b19e25151737 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:39.890004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4c7a44fa-57f3-4484-acec-b19e25151737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:39.906125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d197962-d304-4137-a814-808892985a7a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:39.947915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76815520-810b-4942-85d8-ca413ca5c268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:39.948668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76815520-810b-4942-85d8-ca413ca5c268 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:39.949164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76815520-810b-4942-85d8-ca413ca5c268 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:39.949652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76815520-810b-4942-85d8-ca413ca5c268 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:39.950396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76815520-810b-4942-85d8-ca413ca5c268 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:39.950708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76815520-810b-4942-85d8-ca413ca5c268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:39.952114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76815520-810b-4942-85d8-ca413ca5c268 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:39.961786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 163/327] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:39.967801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76815520-810b-4942-85d8-ca413ca5c268 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:39.969436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:39.976083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 165/328] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:40.078656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76815520-810b-4942-85d8-ca413ca5c268 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:40.080406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:40.116714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff5119-2e46-47af-b0a7-0eb1efdc3712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:40.117365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff5119-2e46-47af-b0a7-0eb1efdc3712 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:40.117648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff5119-2e46-47af-b0a7-0eb1efdc3712 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:40.117925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff5119-2e46-47af-b0a7-0eb1efdc3712 None admin] PATH_INFO: `/v3/users/9300370a5d0345d09dcde553bb85dab2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:40.119100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff5119-2e46-47af-b0a7-0eb1efdc3712 None admin] RBAC: Authorizing `identity:get_user(user_id=9300370a5d0345d09dcde553bb85dab2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:40.120783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff5119-2e46-47af-b0a7-0eb1efdc3712 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:40.122744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 164/329] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:39 2022] GET /identity/v3/users/9300370a5d0345d09dcde553bb85dab2 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:40.128093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a44fa-57f3-4484-acec-b19e25151737 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:40.253797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a44fa-57f3-4484-acec-b19e25151737 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:40.259640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff5119-2e46-47af-b0a7-0eb1efdc3712 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:40.298905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79fe54-7f81-4e8d-badb-36c08efa7952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:40.299909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79fe54-7f81-4e8d-badb-36c08efa7952 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:40.300442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79fe54-7f81-4e8d-badb-36c08efa7952 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:40.300924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79fe54-7f81-4e8d-badb-36c08efa7952 None admin] PATH_INFO: `/v3/system/users/9300370a5d0345d09dcde553bb85dab2/roles/12045ab231c24ebeb9dc0c0b7f6af56a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:40.302581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79fe54-7f81-4e8d-badb-36c08efa7952 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9300370a5d0345d09dcde553bb85dab2, role_id=12045ab231c24ebeb9dc0c0b7f6af56a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:40.304308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79fe54-7f81-4e8d-badb-36c08efa7952 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:40.312889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 166/330] 10.222.0.75 () {58 vars in 1334 bytes} [Fri Aug 19 15:09:40 2022] PUT /identity/v3/system/users/9300370a5d0345d09dcde553bb85dab2/roles/12045ab231c24ebeb9dc0c0b7f6af56a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:09:40.388748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff5119-2e46-47af-b0a7-0eb1efdc3712 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:40.429894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6dffe-3fd9-478e-8f24-5cde7591617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:40.430553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6dffe-3fd9-478e-8f24-5cde7591617f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:40.430833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6dffe-3fd9-478e-8f24-5cde7591617f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:40.431114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6dffe-3fd9-478e-8f24-5cde7591617f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:40.444733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6dffe-3fd9-478e-8f24-5cde7591617f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6ca6dffe-3fd9-478e-8f24-5cde7591617f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:40.451614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:40.452601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:40.453434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 165/331] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:40.459210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79fe54-7f81-4e8d-badb-36c08efa7952 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:40.541506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:40.584345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79fe54-7f81-4e8d-badb-36c08efa7952 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:40.620748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23b967-d978-407c-93e0-4adfa42a90aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:40.621363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23b967-d978-407c-93e0-4adfa42a90aa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:40.621646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23b967-d978-407c-93e0-4adfa42a90aa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:40.621917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23b967-d978-407c-93e0-4adfa42a90aa None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:40.622590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23b967-d978-407c-93e0-4adfa42a90aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:40.622889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23b967-d978-407c-93e0-4adfa42a90aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:40.624359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23b967-d978-407c-93e0-4adfa42a90aa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:40.633405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 167/332] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:40.639785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6dffe-3fd9-478e-8f24-5cde7591617f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:40.687734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:40.774063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6dffe-3fd9-478e-8f24-5cde7591617f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:40.813666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cef13-c536-4d38-870f-10aa2ac9c488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:40.814290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cef13-c536-4d38-870f-10aa2ac9c488 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:40.814514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cef13-c536-4d38-870f-10aa2ac9c488 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:40.814785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cef13-c536-4d38-870f-10aa2ac9c488 None admin] PATH_INFO: `/v3/users/58dc8e8334a04b4f914dfd7c36bd02fc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:40.815938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cef13-c536-4d38-870f-10aa2ac9c488 None admin] RBAC: Authorizing `identity:get_user(user_id=58dc8e8334a04b4f914dfd7c36bd02fc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:40.817533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cef13-c536-4d38-870f-10aa2ac9c488 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:40.819528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 166/333] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:40 2022] GET /identity/v3/users/58dc8e8334a04b4f914dfd7c36bd02fc => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:40.825682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23b967-d978-407c-93e0-4adfa42a90aa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:40.936983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23b967-d978-407c-93e0-4adfa42a90aa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:40.961818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:40.974439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7ed87-966a-4f63-ab2d-e66f2445d193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:40.975105 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7ed87-966a-4f63-ab2d-e66f2445d193 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:40.975717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7ed87-966a-4f63-ab2d-e66f2445d193 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:40.976419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7ed87-966a-4f63-ab2d-e66f2445d193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:40.977173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7ed87-966a-4f63-ab2d-e66f2445d193 None admin] RBAC: Adding query filter params (role.id=a7f0f98b4677401690c354299b049892, user.id=58dc8e8334a04b4f914dfd7c36bd02fc, scope.system=all) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:40.977483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7ed87-966a-4f63-ab2d-e66f2445d193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:40.979027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7ed87-966a-4f63-ab2d-e66f2445d193 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:40.992694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 168/334] 10.222.0.75 () {56 vars in 1308 bytes} [Fri Aug 19 15:09:40 2022] GET /identity/v3/role_assignments?user.id=58dc8e8334a04b4f914dfd7c36bd02fc&scope.system=all&role.id=a7f0f98b4677401690c354299b049892 => generated 510 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:42.169346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f08bb-4178-4413-b14d-dc68aa4869ee None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:42.171505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f08bb-4178-4413-b14d-dc68aa4869ee None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:42.172146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f08bb-4178-4413-b14d-dc68aa4869ee None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:42.174799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 167/335] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:42.180790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d1d2-db78-43ba-8cc3-cf16d01cdb3a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:42.181664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d1d2-db78-43ba-8cc3-cf16d01cdb3a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:42.182139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d1d2-db78-43ba-8cc3-cf16d01cdb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:42.209379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ae23d1d2-db78-43ba-8cc3-cf16d01cdb3a None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:42.210912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ae23d1d2-db78-43ba-8cc3-cf16d01cdb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:42.295409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 169/336] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:42.435009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55928fe-40c9-467f-b768-a226aa7b2f99 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:42.435613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55928fe-40c9-467f-b768-a226aa7b2f99 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:42.436092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55928fe-40c9-467f-b768-a226aa7b2f99 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:42.437959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 168/337] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:42.443321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9d9dd-5e96-4939-9c2e-948fd035d708 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:42.443867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9d9dd-5e96-4939-9c2e-948fd035d708 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:42.444340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9d9dd-5e96-4939-9c2e-948fd035d708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:42.466162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e1b9d9dd-5e96-4939-9c2e-948fd035d708 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:42.467657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e1b9d9dd-5e96-4939-9c2e-948fd035d708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:42.546139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 170/338] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:42.766536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55928fe-40c9-467f-b768-a226aa7b2f99 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:42.876067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9d9dd-5e96-4939-9c2e-948fd035d708 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:42.916010 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:42.925033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55928fe-40c9-467f-b768-a226aa7b2f99 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:42.959968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:42.990473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a0f6-092d-4ece-88aa-497cd7c89c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:42.991109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a0f6-092d-4ece-88aa-497cd7c89c99 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:42.991388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a0f6-092d-4ece-88aa-497cd7c89c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:42.991653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699a0f6-092d-4ece-88aa-497cd7c89c99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:43.018395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699a0f6-092d-4ece-88aa-497cd7c89c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7699a0f6-092d-4ece-88aa-497cd7c89c99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:43.023119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:43.024033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:43.024905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:43.026256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 169/339] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:43.036837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a0f6-092d-4ece-88aa-497cd7c89c99 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:43.087454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9d9dd-5e96-4939-9c2e-948fd035d708 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:43.134723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05380de5-81e6-4d49-a8bb-e8f22fdfeae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:43.135723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380de5-81e6-4d49-a8bb-e8f22fdfeae0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:43.136760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380de5-81e6-4d49-a8bb-e8f22fdfeae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:43.137254 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380de5-81e6-4d49-a8bb-e8f22fdfeae0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:43.153154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05380de5-81e6-4d49-a8bb-e8f22fdfeae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-05380de5-81e6-4d49-a8bb-e8f22fdfeae0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:43.160139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:43.161037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:43.161846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:43.170297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 171/340] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:43.177028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05380de5-81e6-4d49-a8bb-e8f22fdfeae0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:43.258633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699a0f6-092d-4ece-88aa-497cd7c89c99 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:43.299725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a4cd4-a231-4b55-b8da-15603d02c634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:43.300267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a4cd4-a231-4b55-b8da-15603d02c634 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:43.300569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a4cd4-a231-4b55-b8da-15603d02c634 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:43.300841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a4cd4-a231-4b55-b8da-15603d02c634 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:43.301515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a4cd4-a231-4b55-b8da-15603d02c634 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:43.301816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a4cd4-a231-4b55-b8da-15603d02c634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:43.303216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a4cd4-a231-4b55-b8da-15603d02c634 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:43.318026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 170/341] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:43.324351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a4cd4-a231-4b55-b8da-15603d02c634 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:43.355604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05380de5-81e6-4d49-a8bb-e8f22fdfeae0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:43.361858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:43.362667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:43.418279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67455b97-9cde-4d51-8a6d-871885c31ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:43.420213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67455b97-9cde-4d51-8a6d-871885c31ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:43.420535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67455b97-9cde-4d51-8a6d-871885c31ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:43.420811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67455b97-9cde-4d51-8a6d-871885c31ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:43.421498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67455b97-9cde-4d51-8a6d-871885c31ab8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:43.421798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67455b97-9cde-4d51-8a6d-871885c31ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:43.424118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67455b97-9cde-4d51-8a6d-871885c31ab8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:43.442123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 172/342] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:43.447714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67455b97-9cde-4d51-8a6d-871885c31ab8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:43.519804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a4cd4-a231-4b55-b8da-15603d02c634 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:43.569359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb732408-cc41-4a73-af03-c576da51709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:43.570577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb732408-cc41-4a73-af03-c576da51709c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:43.571523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb732408-cc41-4a73-af03-c576da51709c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:43.572143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb732408-cc41-4a73-af03-c576da51709c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:43.594892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67455b97-9cde-4d51-8a6d-871885c31ab8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:43.645851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b442be5-a2d4-412f-a428-35fcae7a7254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:43.646612 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b442be5-a2d4-412f-a428-35fcae7a7254 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:43.647014 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b442be5-a2d4-412f-a428-35fcae7a7254 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:43.647438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b442be5-a2d4-412f-a428-35fcae7a7254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:43.665185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb732408-cc41-4a73-af03-c576da51709c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:43.669072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb732408-cc41-4a73-af03-c576da51709c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:43.700678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:43.705655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:43.764935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b442be5-a2d4-412f-a428-35fcae7a7254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:43.766852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b442be5-a2d4-412f-a428-35fcae7a7254 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:43.801616 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 171/343] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:43.805534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38488-dd7d-484f-a476-dd17e74be84a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:43.805534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38488-dd7d-484f-a476-dd17e74be84a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:43.805534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa38488-dd7d-484f-a476-dd17e74be84a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:43.805534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 172/344] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:43.806087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d215a2-fe67-4c4e-bac2-f5050c0cf437 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:43.806383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d215a2-fe67-4c4e-bac2-f5050c0cf437 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:43.806641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d215a2-fe67-4c4e-bac2-f5050c0cf437 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:43.808478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 173/345] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:43.813272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d215a2-fe67-4c4e-bac2-f5050c0cf437 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:43.924431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 173/346] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:43.925739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83ff5d-cc17-4f0a-baa7-ae45e8ccf1c4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:43.926028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83ff5d-cc17-4f0a-baa7-ae45e8ccf1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:43.926304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83ff5d-cc17-4f0a-baa7-ae45e8ccf1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:43.946995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3f83ff5d-cc17-4f0a-baa7-ae45e8ccf1c4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:43.948191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f83ff5d-cc17-4f0a-baa7-ae45e8ccf1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:43.963505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d215a2-fe67-4c4e-bac2-f5050c0cf437 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:43.993284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:44.015670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482ca4d-ede8-4168-9771-9064abacc205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:44.016130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482ca4d-ede8-4168-9771-9064abacc205 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.016530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482ca4d-ede8-4168-9771-9064abacc205 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.016810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482ca4d-ede8-4168-9771-9064abacc205 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.018048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482ca4d-ede8-4168-9771-9064abacc205 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:44.019703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482ca4d-ede8-4168-9771-9064abacc205 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:44.022678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 174/347] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:43 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:44.023943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:44.024052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372ae40-43c9-4089-8082-b9d75716eedd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.024278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372ae40-43c9-4089-8082-b9d75716eedd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.028439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372ae40-43c9-4089-8082-b9d75716eedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.045733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 174/348] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:44.046821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83ff5d-cc17-4f0a-baa7-ae45e8ccf1c4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:44.056446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f372ae40-43c9-4089-8082-b9d75716eedd None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:44.057709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f372ae40-43c9-4089-8082-b9d75716eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:44.154503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 175/349] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:44.155678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372ae40-43c9-4089-8082-b9d75716eedd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:44.184829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83ff5d-cc17-4f0a-baa7-ae45e8ccf1c4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:44.226437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dc8e5-4d4b-41ef-9e08-20bd98b09758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:44.227077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dc8e5-4d4b-41ef-9e08-20bd98b09758 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.227355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dc8e5-4d4b-41ef-9e08-20bd98b09758 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.227626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dc8e5-4d4b-41ef-9e08-20bd98b09758 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.229005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dc8e5-4d4b-41ef-9e08-20bd98b09758 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:44.231303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dc8e5-4d4b-41ef-9e08-20bd98b09758 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:44.233305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 175/350] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:44 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:44.234409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb89b78-c3fa-4ec4-aba3-c627371ba9da None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.234746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb89b78-c3fa-4ec4-aba3-c627371ba9da None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.235024 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb89b78-c3fa-4ec4-aba3-c627371ba9da None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.236584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 176/351] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:44.239326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb89b78-c3fa-4ec4-aba3-c627371ba9da None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:44.356135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372ae40-43c9-4089-8082-b9d75716eedd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:44.380702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:44.389512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:44.407079 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f27c7-8ce3-4562-ba58-9e37bea1b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:44.408642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f27c7-8ce3-4562-ba58-9e37bea1b26d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.408983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f27c7-8ce3-4562-ba58-9e37bea1b26d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.409258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f27c7-8ce3-4562-ba58-9e37bea1b26d None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.417206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f27c7-8ce3-4562-ba58-9e37bea1b26d None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:44.419512 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f27c7-8ce3-4562-ba58-9e37bea1b26d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:44.421780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 176/352] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:44 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:44.423706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc888a-ed4b-4213-9513-8a40c219405c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.424005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc888a-ed4b-4213-9513-8a40c219405c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.424275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc888a-ed4b-4213-9513-8a40c219405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.451821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb89b78-c3fa-4ec4-aba3-c627371ba9da None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:44.453778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-70bc888a-ed4b-4213-9513-8a40c219405c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:44.455917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-70bc888a-ed4b-4213-9513-8a40c219405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:44.499057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a06a75-a075-42dd-8467-53ba1af02dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:44.499705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a06a75-a075-42dd-8467-53ba1af02dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.499988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a06a75-a075-42dd-8467-53ba1af02dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.500263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a06a75-a075-42dd-8467-53ba1af02dd5 None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.501751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a06a75-a075-42dd-8467-53ba1af02dd5 None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:44.503240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a06a75-a075-42dd-8467-53ba1af02dd5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:44.505939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 177/353] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:44 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:44.513094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a06a75-a075-42dd-8467-53ba1af02dd5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:44.541444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 177/354] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:44.545416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc888a-ed4b-4213-9513-8a40c219405c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:44.649919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a06a75-a075-42dd-8467-53ba1af02dd5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:44.689384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:44.690440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc888a-ed4b-4213-9513-8a40c219405c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:44.696734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b3229-913d-4ee4-a464-7332eab0c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:44.697553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b3229-913d-4ee4-a464-7332eab0c09d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.697980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b3229-913d-4ee4-a464-7332eab0c09d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.698430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b3229-913d-4ee4-a464-7332eab0c09d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.707132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:44.715098 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b3229-913d-4ee4-a464-7332eab0c09d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f51b3229-913d-4ee4-a464-7332eab0c09d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:44.721416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:44.722316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:44.723156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:44.723767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 178/355] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:44.724297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b3229-913d-4ee4-a464-7332eab0c09d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:44.744303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9f764-3bd9-4b27-88a9-a255d69c02cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:44.744967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9f764-3bd9-4b27-88a9-a255d69c02cd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.745874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9f764-3bd9-4b27-88a9-a255d69c02cd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.746149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9f764-3bd9-4b27-88a9-a255d69c02cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.746891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9f764-3bd9-4b27-88a9-a255d69c02cd None admin] RBAC: Adding query filter params (group.id=fd9612958ebd4b67b99222f855b4a660, role.id=a7f0f98b4677401690c354299b049892, scope.project.id=c61f47242a0d40f59b342520453f190d) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:44.747202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9f764-3bd9-4b27-88a9-a255d69c02cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:44.748689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9f764-3bd9-4b27-88a9-a255d69c02cd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:44.756150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 178/356] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:44 2022] GET /identity/v3/role_assignments?group.id=fd9612958ebd4b67b99222f855b4a660&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=a7f0f98b4677401690c354299b049892 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:44.757113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9f764-3bd9-4b27-88a9-a255d69c02cd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:44.887308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b3229-913d-4ee4-a464-7332eab0c09d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:44.891861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9f764-3bd9-4b27-88a9-a255d69c02cd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:44.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae6ac7-d3cf-46fd-b67f-d13074dcbf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:44.934723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae6ac7-d3cf-46fd-b67f-d13074dcbf2b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.935003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae6ac7-d3cf-46fd-b67f-d13074dcbf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.935276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae6ac7-d3cf-46fd-b67f-d13074dcbf2b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.937787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22709148-fa9b-4402-8482-566e68e1bea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:44.938402 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22709148-fa9b-4402-8482-566e68e1bea8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.938686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22709148-fa9b-4402-8482-566e68e1bea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.938963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22709148-fa9b-4402-8482-566e68e1bea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.939697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22709148-fa9b-4402-8482-566e68e1bea8 None admin] RBAC: Adding query filter params (group.id=fd9612958ebd4b67b99222f855b4a660, role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=c61f47242a0d40f59b342520453f190d) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:44.940009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22709148-fa9b-4402-8482-566e68e1bea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:44.941679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22709148-fa9b-4402-8482-566e68e1bea8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:44.948443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae6ac7-d3cf-46fd-b67f-d13074dcbf2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:44.950963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 179/357] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:44 2022] GET /identity/v3/role_assignments?group.id=fd9612958ebd4b67b99222f855b4a660&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=048908af57db471987eb7f2ec21e0304 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:44.952554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f0f0-308f-4101-b223-4b377de5a0dd None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:44.953046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f0f0-308f-4101-b223-4b377de5a0dd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:44.953555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3f0f0-308f-4101-b223-4b377de5a0dd None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-eaae6ac7-d3cf-46fd-b67f-d13074dcbf2b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:44.954876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:44.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:44.956601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 179/358] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:44.957334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae6ac7-d3cf-46fd-b67f-d13074dcbf2b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:44.962502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 180/359] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:44.964352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3f0f0-308f-4101-b223-4b377de5a0dd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:45.020761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:45.048856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:45.100836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3f0f0-308f-4101-b223-4b377de5a0dd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:45.105249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae6ac7-d3cf-46fd-b67f-d13074dcbf2b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:45.159681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf55ca6-4a7e-4d35-8966-171bd5aab132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:45.160271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c5709-9ec5-4e9d-beb4-91fe798e4feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:45.160984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5709-9ec5-4e9d-beb4-91fe798e4feb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:45.161181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55ca6-4a7e-4d35-8966-171bd5aab132 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:45.161278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5709-9ec5-4e9d-beb4-91fe798e4feb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:45.161516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5709-9ec5-4e9d-beb4-91fe798e4feb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:45.162000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55ca6-4a7e-4d35-8966-171bd5aab132 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:45.162463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf55ca6-4a7e-4d35-8966-171bd5aab132 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:45.163402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf55ca6-4a7e-4d35-8966-171bd5aab132 None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:45.164103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf55ca6-4a7e-4d35-8966-171bd5aab132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:45.166108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf55ca6-4a7e-4d35-8966-171bd5aab132 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:45.176371 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c5709-9ec5-4e9d-beb4-91fe798e4feb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:45.180216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 180/360] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:45.181442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf55ca6-4a7e-4d35-8966-171bd5aab132 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-501c5709-9ec5-4e9d-beb4-91fe798e4feb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:45.182132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:45.183120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:45.183922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 181/361] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:45.185304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e578899-8eda-4db0-84b5-b0d92babe465 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:45.185474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e578899-8eda-4db0-84b5-b0d92babe465 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:45.185718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e578899-8eda-4db0-84b5-b0d92babe465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:45.213610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9e578899-8eda-4db0-84b5-b0d92babe465 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:45.215712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e578899-8eda-4db0-84b5-b0d92babe465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:45.314531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 182/362] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:45.315334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e578899-8eda-4db0-84b5-b0d92babe465 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:45.318379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf55ca6-4a7e-4d35-8966-171bd5aab132 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:45.347541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:45.380644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:45.384787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed695c4-6b30-45c2-8039-5ec5640ea9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:45.385407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed695c4-6b30-45c2-8039-5ec5640ea9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:45.385686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed695c4-6b30-45c2-8039-5ec5640ea9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:45.389246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed695c4-6b30-45c2-8039-5ec5640ea9a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:45.389981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed695c4-6b30-45c2-8039-5ec5640ea9a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:45.390478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed695c4-6b30-45c2-8039-5ec5640ea9a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:45.392428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed695c4-6b30-45c2-8039-5ec5640ea9a0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:45.411929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 181/363] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:45.412813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed695c4-6b30-45c2-8039-5ec5640ea9a0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:45.510305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e578899-8eda-4db0-84b5-b0d92babe465 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:45.555897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32738b0a-592e-41b8-b0e7-dfd73947dddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:45.556579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32738b0a-592e-41b8-b0e7-dfd73947dddf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:45.556863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32738b0a-592e-41b8-b0e7-dfd73947dddf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:45.557133 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32738b0a-592e-41b8-b0e7-dfd73947dddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:45.647719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32738b0a-592e-41b8-b0e7-dfd73947dddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:45.649533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32738b0a-592e-41b8-b0e7-dfd73947dddf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:45.670119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed695c4-6b30-45c2-8039-5ec5640ea9a0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:45.686267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:45.718453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b09ed-1a79-4eee-afc3-3d62f5957d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:45.718453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b09ed-1a79-4eee-afc3-3d62f5957d3f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:45.718717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b09ed-1a79-4eee-afc3-3d62f5957d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:45.718878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b09ed-1a79-4eee-afc3-3d62f5957d3f None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:45.719739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b09ed-1a79-4eee-afc3-3d62f5957d3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:45.720115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b09ed-1a79-4eee-afc3-3d62f5957d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:45.721996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b09ed-1a79-4eee-afc3-3d62f5957d3f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:45.737305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 182/364] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:45.737305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b09ed-1a79-4eee-afc3-3d62f5957d3f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:45.784915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:45.786425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 183/365] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:45.787100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32738b0a-592e-41b8-b0e7-dfd73947dddf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:45.876777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b09ed-1a79-4eee-afc3-3d62f5957d3f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:45.908517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32738b0a-592e-41b8-b0e7-dfd73947dddf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:45.918923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78759748-3d09-4696-8b95-bdcb75ee95e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:45.919533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78759748-3d09-4696-8b95-bdcb75ee95e2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:45.919860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78759748-3d09-4696-8b95-bdcb75ee95e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:45.920134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78759748-3d09-4696-8b95-bdcb75ee95e2 None admin] PATH_INFO: `/v3/users/9300370a5d0345d09dcde553bb85dab2` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:45.922399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78759748-3d09-4696-8b95-bdcb75ee95e2 None admin] RBAC: Authorizing `identity:get_user(user_id=9300370a5d0345d09dcde553bb85dab2)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:45.924005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78759748-3d09-4696-8b95-bdcb75ee95e2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:45.925776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 183/366] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:09:45 2022] GET /identity/v3/users/9300370a5d0345d09dcde553bb85dab2 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:45.926543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78759748-3d09-4696-8b95-bdcb75ee95e2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:45.951563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:45.962589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c2dc2-8cb7-432a-b37c-ec5b6a740201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:45.963687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c2dc2-8cb7-432a-b37c-ec5b6a740201 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:45.965049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c2dc2-8cb7-432a-b37c-ec5b6a740201 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:45.965684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c2dc2-8cb7-432a-b37c-ec5b6a740201 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:45.979673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c2dc2-8cb7-432a-b37c-ec5b6a740201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-274c2dc2-8cb7-432a-b37c-ec5b6a740201 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:45.986152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:45.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:45.987834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:45.990177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 184/367] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:09:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:45.995910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c2dc2-8cb7-432a-b37c-ec5b6a740201 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:46.059686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78759748-3d09-4696-8b95-bdcb75ee95e2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:46.076880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:46.102468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f74d18-d8d6-4605-8051-f7ea83ca8623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:46.103221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f74d18-d8d6-4605-8051-f7ea83ca8623 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:46.105052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f74d18-d8d6-4605-8051-f7ea83ca8623 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:46.105052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f74d18-d8d6-4605-8051-f7ea83ca8623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:46.133664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c2dc2-8cb7-432a-b37c-ec5b6a740201 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:46.181252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae49a52-8809-4024-89a8-e8ddfbd0d5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:46.182192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae49a52-8809-4024-89a8-e8ddfbd0d5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:46.183352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae49a52-8809-4024-89a8-e8ddfbd0d5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:46.183799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae49a52-8809-4024-89a8-e8ddfbd0d5d4 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:46.185675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae49a52-8809-4024-89a8-e8ddfbd0d5d4 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:46.187497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae49a52-8809-4024-89a8-e8ddfbd0d5d4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:46.190557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 185/368] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:45 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:46.190687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f74d18-d8d6-4605-8051-f7ea83ca8623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:46.191634 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae49a52-8809-4024-89a8-e8ddfbd0d5d4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:46.192435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f74d18-d8d6-4605-8051-f7ea83ca8623 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:46.261314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:46.315703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae49a52-8809-4024-89a8-e8ddfbd0d5d4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:46.328920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 184/369] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:46.330331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f74d18-d8d6-4605-8051-f7ea83ca8623 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:46.352773 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:46.369866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84c3d6-d0d2-4997-a56c-3b5c7e6337d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:46.371292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84c3d6-d0d2-4997-a56c-3b5c7e6337d6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:46.371573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84c3d6-d0d2-4997-a56c-3b5c7e6337d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:46.371847 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84c3d6-d0d2-4997-a56c-3b5c7e6337d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:46.372660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84c3d6-d0d2-4997-a56c-3b5c7e6337d6 None admin] RBAC: Adding query filter params (role.id=12045ab231c24ebeb9dc0c0b7f6af56a, user.id=9300370a5d0345d09dcde553bb85dab2, scope.system=all) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:46.372973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84c3d6-d0d2-4997-a56c-3b5c7e6337d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:46.375453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84c3d6-d0d2-4997-a56c-3b5c7e6337d6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:46.390610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 186/370] 10.222.0.75 () {56 vars in 1308 bytes} [Fri Aug 19 15:09:46 2022] GET /identity/v3/role_assignments?user.id=9300370a5d0345d09dcde553bb85dab2&scope.system=all&role.id=12045ab231c24ebeb9dc0c0b7f6af56a => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:46.391763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84c3d6-d0d2-4997-a56c-3b5c7e6337d6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:46.471805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f74d18-d8d6-4605-8051-f7ea83ca8623 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:46.529428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84c3d6-d0d2-4997-a56c-3b5c7e6337d6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:46.549795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:46.571952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbb533-c8ab-4908-b2a2-d5b1b99a9063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:46.572550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbb533-c8ab-4908-b2a2-d5b1b99a9063 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:46.572938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbb533-c8ab-4908-b2a2-d5b1b99a9063 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:46.573352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbb533-c8ab-4908-b2a2-d5b1b99a9063 None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:46.574593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbb533-c8ab-4908-b2a2-d5b1b99a9063 None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:46.575259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d75942-23e0-4b41-9bbe-e7889c6d0a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:46.575951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75942-23e0-4b41-9bbe-e7889c6d0a3c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:46.576051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbb533-c8ab-4908-b2a2-d5b1b99a9063 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:46.576320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75942-23e0-4b41-9bbe-e7889c6d0a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:46.576695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75942-23e0-4b41-9bbe-e7889c6d0a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:46.577506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d75942-23e0-4b41-9bbe-e7889c6d0a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:46.577675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 187/371] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:46 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:46.577920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d75942-23e0-4b41-9bbe-e7889c6d0a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:46.578428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbb533-c8ab-4908-b2a2-d5b1b99a9063 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:46.579475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d75942-23e0-4b41-9bbe-e7889c6d0a3c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:46.601291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 185/372] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:46.602735 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d75942-23e0-4b41-9bbe-e7889c6d0a3c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:46.690551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:46.719795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbb533-c8ab-4908-b2a2-d5b1b99a9063 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:46.731653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d75942-23e0-4b41-9bbe-e7889c6d0a3c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:46.759193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bac78-f2d2-4780-bf27-9232bacb29ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:46.760220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bac78-f2d2-4780-bf27-9232bacb29ec None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:46.761285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bac78-f2d2-4780-bf27-9232bacb29ec None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:46.761745 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bac78-f2d2-4780-bf27-9232bacb29ec None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:46.763108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bac78-f2d2-4780-bf27-9232bacb29ec None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:46.764909 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bac78-f2d2-4780-bf27-9232bacb29ec None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:46.766903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 188/373] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:46 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:46.767890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bac78-f2d2-4780-bf27-9232bacb29ec None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:46.770415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fb3c6-509f-4520-b4d6-d02f76b096c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:46.771146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fb3c6-509f-4520-b4d6-d02f76b096c8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:46.771432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fb3c6-509f-4520-b4d6-d02f76b096c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:46.771704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fb3c6-509f-4520-b4d6-d02f76b096c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:46.773198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fb3c6-509f-4520-b4d6-d02f76b096c8 None admin] RBAC: Adding query filter params (group.id=fd9612958ebd4b67b99222f855b4a660, role.id=a7f0f98b4677401690c354299b049892, scope.project.id=282dfae79c6e4e20aac9334bc958b580) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:46.773198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fb3c6-509f-4520-b4d6-d02f76b096c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:46.774377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36fb3c6-509f-4520-b4d6-d02f76b096c8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:46.781636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 186/374] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:46 2022] GET /identity/v3/role_assignments?group.id=fd9612958ebd4b67b99222f855b4a660&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=a7f0f98b4677401690c354299b049892 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:46.782579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fb3c6-509f-4520-b4d6-d02f76b096c8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:46.836229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:46.893248 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bac78-f2d2-4780-bf27-9232bacb29ec None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:46.939474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12880580-a7f2-49b5-b164-5ae97b3984a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:46.940596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12880580-a7f2-49b5-b164-5ae97b3984a5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:46.942679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fb3c6-509f-4520-b4d6-d02f76b096c8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:46.943179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12880580-a7f2-49b5-b164-5ae97b3984a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:46.943635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12880580-a7f2-49b5-b164-5ae97b3984a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.008538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:47.029725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4494-3b4b-4f78-97a0-e2d5dbffb14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:47.030349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4494-3b4b-4f78-97a0-e2d5dbffb14a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.030678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4494-3b4b-4f78-97a0-e2d5dbffb14a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.030948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4494-3b4b-4f78-97a0-e2d5dbffb14a None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.032096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf4494-3b4b-4f78-97a0-e2d5dbffb14a None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:47.033578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf4494-3b4b-4f78-97a0-e2d5dbffb14a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:47.044693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 187/375] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:46 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:47.046606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4494-3b4b-4f78-97a0-e2d5dbffb14a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:47.095151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12880580-a7f2-49b5-b164-5ae97b3984a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:47.096819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12880580-a7f2-49b5-b164-5ae97b3984a5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:47.173370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:47.209235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4494-3b4b-4f78-97a0-e2d5dbffb14a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:47.250717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 189/376] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:47.251572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435b502-541d-411f-a7cd-c7853fe987c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:47.252186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435b502-541d-411f-a7cd-c7853fe987c8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.252501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435b502-541d-411f-a7cd-c7853fe987c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.252775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435b502-541d-411f-a7cd-c7853fe987c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.253289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35615df9-6091-4358-bbb6-f0901c6a25f2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.253512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435b502-541d-411f-a7cd-c7853fe987c8 None admin] RBAC: Adding query filter params (group.id=fd9612958ebd4b67b99222f855b4a660, role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=282dfae79c6e4e20aac9334bc958b580) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:47.253818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435b502-541d-411f-a7cd-c7853fe987c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:47.254115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35615df9-6091-4358-bbb6-f0901c6a25f2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.254567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35615df9-6091-4358-bbb6-f0901c6a25f2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.255263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435b502-541d-411f-a7cd-c7853fe987c8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:47.262050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 190/377] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:47.263107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35615df9-6091-4358-bbb6-f0901c6a25f2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:47.275585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 188/378] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:47 2022] GET /identity/v3/role_assignments?group.id=fd9612958ebd4b67b99222f855b4a660&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=048908af57db471987eb7f2ec21e0304 => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:47.277462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64454416-a2ee-4989-a8e5-6451e527da23 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64454416-a2ee-4989-a8e5-6451e527da23 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.283094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64454416-a2ee-4989-a8e5-6451e527da23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.303502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-64454416-a2ee-4989-a8e5-6451e527da23 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:47.305089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-64454416-a2ee-4989-a8e5-6451e527da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:47.377907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:47.395283 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 189/379] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:47.395467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35615df9-6091-4358-bbb6-f0901c6a25f2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:47.445394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def27b4a-8fac-421f-9e4b-afd02a15b537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:47.446037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27b4a-8fac-421f-9e4b-afd02a15b537 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.446316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27b4a-8fac-421f-9e4b-afd02a15b537 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.446587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27b4a-8fac-421f-9e4b-afd02a15b537 None admin] PATH_INFO: `/v3/projects/a9a96491d392440daeaecef9eef8a152` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.447721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def27b4a-8fac-421f-9e4b-afd02a15b537 None admin] RBAC: Authorizing `identity:get_project(project_id=a9a96491d392440daeaecef9eef8a152)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:47.450073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def27b4a-8fac-421f-9e4b-afd02a15b537 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:47.451707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 191/380] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:47 2022] GET /identity/v3/projects/a9a96491d392440daeaecef9eef8a152 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:47.457340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64454416-a2ee-4989-a8e5-6451e527da23 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:47.493217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd466c-4723-4b5f-97ca-206949c78243 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.493217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd466c-4723-4b5f-97ca-206949c78243 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.493217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd466c-4723-4b5f-97ca-206949c78243 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.493758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 192/381] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:47.499649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604446a0-5c39-46b5-af4c-11e654f5c8d0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.499941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604446a0-5c39-46b5-af4c-11e654f5c8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.500201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604446a0-5c39-46b5-af4c-11e654f5c8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.533217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-604446a0-5c39-46b5-af4c-11e654f5c8d0 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:47.534686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-604446a0-5c39-46b5-af4c-11e654f5c8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:47.590280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:47.593832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64454416-a2ee-4989-a8e5-6451e527da23 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:47.633536 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aee9b6-f70c-4a5e-bbf3-ee75b44d7b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:47.634459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aee9b6-f70c-4a5e-bbf3-ee75b44d7b83 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.634921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aee9b6-f70c-4a5e-bbf3-ee75b44d7b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.635371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aee9b6-f70c-4a5e-bbf3-ee75b44d7b83 None admin] PATH_INFO: `/v3/groups/d2ec549a5352455d8256184a909f0a1e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.637857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 193/382] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:47.645163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aee9b6-f70c-4a5e-bbf3-ee75b44d7b83 None admin] RBAC: Authorizing `identity:get_group(group_id=d2ec549a5352455d8256184a909f0a1e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:47.646639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aee9b6-f70c-4a5e-bbf3-ee75b44d7b83 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:47.648622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 190/383] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:47 2022] GET /identity/v3/groups/d2ec549a5352455d8256184a909f0a1e => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:47.654580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604446a0-5c39-46b5-af4c-11e654f5c8d0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:47.751911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aee9b6-f70c-4a5e-bbf3-ee75b44d7b83 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:47.773257 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604446a0-5c39-46b5-af4c-11e654f5c8d0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:47.813672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72821343-7c45-48b3-ba42-5dccf68bbe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:47.814708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72821343-7c45-48b3-ba42-5dccf68bbe14 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.815192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72821343-7c45-48b3-ba42-5dccf68bbe14 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.815690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72821343-7c45-48b3-ba42-5dccf68bbe14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.816657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72821343-7c45-48b3-ba42-5dccf68bbe14 None admin] RBAC: Adding query filter params (group.id=d2ec549a5352455d8256184a909f0a1e, role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=a9a96491d392440daeaecef9eef8a152) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:47.817172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72821343-7c45-48b3-ba42-5dccf68bbe14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:47.818901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72821343-7c45-48b3-ba42-5dccf68bbe14 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:47.826064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 194/384] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:47 2022] GET /identity/v3/role_assignments?group.id=d2ec549a5352455d8256184a909f0a1e&scope.project.id=a9a96491d392440daeaecef9eef8a152&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:47.850362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:47.919969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aee9b6-f70c-4a5e-bbf3-ee75b44d7b83 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:47.964525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22dd32-065a-485a-b332-d5939e3bf732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:47.965177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22dd32-065a-485a-b332-d5939e3bf732 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:47.965607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22dd32-065a-485a-b332-d5939e3bf732 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:47.965890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22dd32-065a-485a-b332-d5939e3bf732 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:47.977355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22dd32-065a-485a-b332-d5939e3bf732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:47.985258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72821343-7c45-48b3-ba42-5dccf68bbe14 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4b22dd32-065a-485a-b332-d5939e3bf732 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:47.985924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:47.986873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:47.987727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 191/385] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:48.005208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22dd32-065a-485a-b332-d5939e3bf732 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:48.014336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:48.117321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72821343-7c45-48b3-ba42-5dccf68bbe14 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:48.129210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22dd32-065a-485a-b332-d5939e3bf732 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:48.148089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:48.168414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f2084-3b27-4a65-b716-12dd4c13a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:48.168414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f2084-3b27-4a65-b716-12dd4c13a69b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:48.168414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f2084-3b27-4a65-b716-12dd4c13a69b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:48.168414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f2084-3b27-4a65-b716-12dd4c13a69b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:48.173077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd826984-3b81-46bf-bbd0-1ea16f2729ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:48.173629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd826984-3b81-46bf-bbd0-1ea16f2729ac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:48.173911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd826984-3b81-46bf-bbd0-1ea16f2729ac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:48.174185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd826984-3b81-46bf-bbd0-1ea16f2729ac None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:48.174861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd826984-3b81-46bf-bbd0-1ea16f2729ac None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:48.175165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd826984-3b81-46bf-bbd0-1ea16f2729ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:48.176602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd826984-3b81-46bf-bbd0-1ea16f2729ac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:48.187905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 192/386] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:48.193727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd826984-3b81-46bf-bbd0-1ea16f2729ac None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:48.194144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f2084-3b27-4a65-b716-12dd4c13a69b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-857f2084-3b27-4a65-b716-12dd4c13a69b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:48.199614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:48.200537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:48.201491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 195/387] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:48.213522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f2084-3b27-4a65-b716-12dd4c13a69b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:48.326298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd826984-3b81-46bf-bbd0-1ea16f2729ac None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:48.356142 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:48.378426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4350fa-e504-4cd1-b4a2-f8a0d7967c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:48.379071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4350fa-e504-4cd1-b4a2-f8a0d7967c4a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:48.379352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4350fa-e504-4cd1-b4a2-f8a0d7967c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:48.379620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4350fa-e504-4cd1-b4a2-f8a0d7967c4a None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:48.383468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4350fa-e504-4cd1-b4a2-f8a0d7967c4a None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:48.386865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4350fa-e504-4cd1-b4a2-f8a0d7967c4a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:48.388541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 193/388] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:48 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:48.394693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4350fa-e504-4cd1-b4a2-f8a0d7967c4a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:48.408003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f2084-3b27-4a65-b716-12dd4c13a69b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:48.471229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da02bd-3806-4077-8331-5e0f0e9f8afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:48.471769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da02bd-3806-4077-8331-5e0f0e9f8afa None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:48.472047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da02bd-3806-4077-8331-5e0f0e9f8afa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:48.472321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da02bd-3806-4077-8331-5e0f0e9f8afa None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:48.473026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da02bd-3806-4077-8331-5e0f0e9f8afa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:48.473333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da02bd-3806-4077-8331-5e0f0e9f8afa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:48.474730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da02bd-3806-4077-8331-5e0f0e9f8afa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:48.479295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:48.491734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 196/389] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:48.499615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da02bd-3806-4077-8331-5e0f0e9f8afa None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:48.578807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4350fa-e504-4cd1-b4a2-f8a0d7967c4a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:48.654039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e58f-8feb-41b6-b175-9a551710d526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:48.654688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da02bd-3806-4077-8331-5e0f0e9f8afa None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:48.655615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e58f-8feb-41b6-b175-9a551710d526 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:48.656158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e58f-8feb-41b6-b175-9a551710d526 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:48.656757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e58f-8feb-41b6-b175-9a551710d526 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:48.658962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1e58f-8feb-41b6-b175-9a551710d526 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:48.660718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1e58f-8feb-41b6-b175-9a551710d526 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:48.663030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 194/390] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:48 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:48.671269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e58f-8feb-41b6-b175-9a551710d526 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:48.693672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042694d-8a62-42ef-beb3-ae1a4d5bcd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:48.694367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042694d-8a62-42ef-beb3-ae1a4d5bcd38 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:48.694649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042694d-8a62-42ef-beb3-ae1a4d5bcd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:48.694975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042694d-8a62-42ef-beb3-ae1a4d5bcd38 None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:48.696127 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042694d-8a62-42ef-beb3-ae1a4d5bcd38 None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:48.697618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042694d-8a62-42ef-beb3-ae1a4d5bcd38 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:48.704576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 197/391] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:48 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:48.705948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042694d-8a62-42ef-beb3-ae1a4d5bcd38 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:48.710192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:48.822525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042694d-8a62-42ef-beb3-ae1a4d5bcd38 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:48.849848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e58f-8feb-41b6-b175-9a551710d526 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:48.864803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba04ad-b9e9-484e-84fd-10f841ba9d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:48.865517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba04ad-b9e9-484e-84fd-10f841ba9d92 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:48.865798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba04ad-b9e9-484e-84fd-10f841ba9d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:48.866186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba04ad-b9e9-484e-84fd-10f841ba9d92 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:48.867394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba04ad-b9e9-484e-84fd-10f841ba9d92 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:48.869044 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba04ad-b9e9-484e-84fd-10f841ba9d92 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:48.874645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 198/392] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:48 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:48.875756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:48.877764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba04ad-b9e9-484e-84fd-10f841ba9d92 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:48.913190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc842b3-328b-465f-b41a-7274877f085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:48.913969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc842b3-328b-465f-b41a-7274877f085c None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:48.914259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc842b3-328b-465f-b41a-7274877f085c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:48.914541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc842b3-328b-465f-b41a-7274877f085c None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d/groups/fd9612958ebd4b67b99222f855b4a660/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:48.917842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc842b3-328b-465f-b41a-7274877f085c None admin] RBAC: Authorizing `identity:create_grant(project_id=c61f47242a0d40f59b342520453f190d, group_id=fd9612958ebd4b67b99222f855b4a660, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:48.919380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc842b3-328b-465f-b41a-7274877f085c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:48.946802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 195/393] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Aug 19 15:09:48 2022] PUT /identity/v3/projects/c61f47242a0d40f59b342520453f190d/groups/fd9612958ebd4b67b99222f855b4a660/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:09:48.961857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:48.997526 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba04ad-b9e9-484e-84fd-10f841ba9d92 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:49.036623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd88dd-161a-4387-9091-ceaf428f8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:49.037315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd88dd-161a-4387-9091-ceaf428f8e46 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:49.037677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd88dd-161a-4387-9091-ceaf428f8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:49.037960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd88dd-161a-4387-9091-ceaf428f8e46 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d/groups/fd9612958ebd4b67b99222f855b4a660/roles/048908af57db471987eb7f2ec21e0304` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:49.040253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd88dd-161a-4387-9091-ceaf428f8e46 None admin] RBAC: Authorizing `identity:create_grant(project_id=c61f47242a0d40f59b342520453f190d, group_id=fd9612958ebd4b67b99222f855b4a660, role_id=048908af57db471987eb7f2ec21e0304)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:49.041939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd88dd-161a-4387-9091-ceaf428f8e46 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:49.064570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 199/394] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Aug 19 15:09:48 2022] PUT /identity/v3/projects/c61f47242a0d40f59b342520453f190d/groups/fd9612958ebd4b67b99222f855b4a660/roles/048908af57db471987eb7f2ec21e0304 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:09:49.354846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c87106-1b27-48d9-adcf-41cc5b788838 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:49.355375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c87106-1b27-48d9-adcf-41cc5b788838 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:49.355375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c87106-1b27-48d9-adcf-41cc5b788838 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:49.357039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 196/395] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:49.362541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77beeff-2a7d-47ea-a194-acd47af3a987 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:49.363107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77beeff-2a7d-47ea-a194-acd47af3a987 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:49.363580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77beeff-2a7d-47ea-a194-acd47af3a987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:49.388753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b77beeff-2a7d-47ea-a194-acd47af3a987 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:49.390279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b77beeff-2a7d-47ea-a194-acd47af3a987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:49.479100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 200/396] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:49.923997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4657ad-8002-4b41-8299-b03f1d1756e8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:49.924525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4657ad-8002-4b41-8299-b03f1d1756e8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:49.924525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4657ad-8002-4b41-8299-b03f1d1756e8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:49.926094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 197/397] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:49.931367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435b18a-e2b0-4b0e-824b-fbd5c19ee686 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:49.931901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435b18a-e2b0-4b0e-824b-fbd5c19ee686 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:49.932382 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a435b18a-e2b0-4b0e-824b-fbd5c19ee686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:49.952700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a435b18a-e2b0-4b0e-824b-fbd5c19ee686 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:49.954199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a435b18a-e2b0-4b0e-824b-fbd5c19ee686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:50.017302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:50.032156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4657ad-8002-4b41-8299-b03f1d1756e8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:50.036317 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 201/398] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:50.170120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4657ad-8002-4b41-8299-b03f1d1756e8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:50.205986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f15f0e-936d-4166-a430-99c931621a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:50.206934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f15f0e-936d-4166-a430-99c931621a5a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:50.207415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f15f0e-936d-4166-a430-99c931621a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:50.207920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f15f0e-936d-4166-a430-99c931621a5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:50.219486 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f15f0e-936d-4166-a430-99c931621a5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a7f15f0e-936d-4166-a430-99c931621a5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:50.224629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:50.225519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:50.226337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:50.229074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 198/399] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:50.235804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435b18a-e2b0-4b0e-824b-fbd5c19ee686 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:50.366463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a435b18a-e2b0-4b0e-824b-fbd5c19ee686 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:50.411831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac8967-f46a-4d76-9e4a-fcc4a7416ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:50.412472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8967-f46a-4d76-9e4a-fcc4a7416ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:50.412757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8967-f46a-4d76-9e4a-fcc4a7416ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:50.413034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8967-f46a-4d76-9e4a-fcc4a7416ea6 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:50.413716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac8967-f46a-4d76-9e4a-fcc4a7416ea6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:50.414021 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac8967-f46a-4d76-9e4a-fcc4a7416ea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:50.415413 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac8967-f46a-4d76-9e4a-fcc4a7416ea6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:50.429506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 202/400] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:50.435040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f15f0e-936d-4166-a430-99c931621a5a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:50.494951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:50.496108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac8967-f46a-4d76-9e4a-fcc4a7416ea6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:50.578624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f15f0e-936d-4166-a430-99c931621a5a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:50.587196 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:50.619711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369359e-6a7b-4e5b-8ca7-9be61a7ab4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:50.625066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369359e-6a7b-4e5b-8ca7-9be61a7ab4ab None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:50.625512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369359e-6a7b-4e5b-8ca7-9be61a7ab4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:50.625960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369359e-6a7b-4e5b-8ca7-9be61a7ab4ab None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:50.627396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369359e-6a7b-4e5b-8ca7-9be61a7ab4ab None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:50.629180 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d369359e-6a7b-4e5b-8ca7-9be61a7ab4ab None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:50.631087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 199/401] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:50 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:50.637144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369359e-6a7b-4e5b-8ca7-9be61a7ab4ab None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:50.664930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac8967-f46a-4d76-9e4a-fcc4a7416ea6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:50.700869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24299a68-d81d-46b5-b802-9e3d55847707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:50.701613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24299a68-d81d-46b5-b802-9e3d55847707 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:50.701898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24299a68-d81d-46b5-b802-9e3d55847707 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:50.702176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24299a68-d81d-46b5-b802-9e3d55847707 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:50.712898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24299a68-d81d-46b5-b802-9e3d55847707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-24299a68-d81d-46b5-b802-9e3d55847707 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:50.719501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:50.720672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:50.721463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 203/402] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:50.722172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb197-be91-4031-b81f-e88a02678266 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:50.722172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb197-be91-4031-b81f-e88a02678266 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:50.722290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb197-be91-4031-b81f-e88a02678266 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:50.723520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 204/403] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:50.728671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb197-be91-4031-b81f-e88a02678266 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:50.757473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369359e-6a7b-4e5b-8ca7-9be61a7ab4ab None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:50.761345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:50.797713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15400314-adeb-4640-b79e-45c9cfcb54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:50.798613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15400314-adeb-4640-b79e-45c9cfcb54d5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:50.799051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15400314-adeb-4640-b79e-45c9cfcb54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:50.799475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15400314-adeb-4640-b79e-45c9cfcb54d5 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:50.800958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15400314-adeb-4640-b79e-45c9cfcb54d5 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:50.802711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15400314-adeb-4640-b79e-45c9cfcb54d5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:50.804901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 200/404] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:50 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:50.806351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7068f17-af9c-48f0-b266-8e6d59988089 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:50.806805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7068f17-af9c-48f0-b266-8e6d59988089 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:50.807273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7068f17-af9c-48f0-b266-8e6d59988089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:50.827437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e7068f17-af9c-48f0-b266-8e6d59988089 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:50.829116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e7068f17-af9c-48f0-b266-8e6d59988089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:50.841518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb197-be91-4031-b81f-e88a02678266 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:50.855846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:50.878291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c952-a41f-4439-8990-4b0a5b60abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:50.878997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c952-a41f-4439-8990-4b0a5b60abed None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:50.879339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c952-a41f-4439-8990-4b0a5b60abed None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:50.879686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c952-a41f-4439-8990-4b0a5b60abed None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:50.880563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8c952-a41f-4439-8990-4b0a5b60abed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:50.880940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8c952-a41f-4439-8990-4b0a5b60abed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:50.882518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8c952-a41f-4439-8990-4b0a5b60abed None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:50.892404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 205/405] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:50.893463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c952-a41f-4439-8990-4b0a5b60abed None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:50.910867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 201/406] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:50.912194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7068f17-af9c-48f0-b266-8e6d59988089 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:51.024429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7068f17-af9c-48f0-b266-8e6d59988089 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:51.028750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:51.067403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340272d9-8d6e-41d5-9252-5213a822ca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:51.067605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c952-a41f-4439-8990-4b0a5b60abed None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:51.068554 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340272d9-8d6e-41d5-9252-5213a822ca76 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.069148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340272d9-8d6e-41d5-9252-5213a822ca76 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.069728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340272d9-8d6e-41d5-9252-5213a822ca76 None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.071423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340272d9-8d6e-41d5-9252-5213a822ca76 None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:51.073208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340272d9-8d6e-41d5-9252-5213a822ca76 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:51.075677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 202/407] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:50 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:51.081360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340272d9-8d6e-41d5-9252-5213a822ca76 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:51.106969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24718d75-a41d-4336-b8ba-d3bb246cebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:51.107665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24718d75-a41d-4336-b8ba-d3bb246cebea None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.107931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24718d75-a41d-4336-b8ba-d3bb246cebea None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.108208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24718d75-a41d-4336-b8ba-d3bb246cebea None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580/groups/fd9612958ebd4b67b99222f855b4a660/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.111122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24718d75-a41d-4336-b8ba-d3bb246cebea None admin] RBAC: Authorizing `identity:create_grant(project_id=282dfae79c6e4e20aac9334bc958b580, group_id=fd9612958ebd4b67b99222f855b4a660, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:51.112619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24718d75-a41d-4336-b8ba-d3bb246cebea None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:51.139615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 206/408] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Aug 19 15:09:50 2022] PUT /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580/groups/fd9612958ebd4b67b99222f855b4a660/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:09:51.244047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340272d9-8d6e-41d5-9252-5213a822ca76 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:51.289750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e5bbb-ea46-4beb-a6aa-26f405dfd4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:51.290322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e5bbb-ea46-4beb-a6aa-26f405dfd4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.290605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e5bbb-ea46-4beb-a6aa-26f405dfd4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.290879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e5bbb-ea46-4beb-a6aa-26f405dfd4b0 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.296086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e5bbb-ea46-4beb-a6aa-26f405dfd4b0 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:51.298360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e5bbb-ea46-4beb-a6aa-26f405dfd4b0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:51.300580 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 203/409] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:51 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:51.309663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24718d75-a41d-4336-b8ba-d3bb246cebea None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:51.349707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:51.377290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e5bbb-ea46-4beb-a6aa-26f405dfd4b0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:51.444764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:51.463070 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24718d75-a41d-4336-b8ba-d3bb246cebea None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:51.516736 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ec61e-5d1d-454a-a7c4-102536495bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:51.517746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ec61e-5d1d-454a-a7c4-102536495bdc None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.518169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ec61e-5d1d-454a-a7c4-102536495bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.518590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ec61e-5d1d-454a-a7c4-102536495bdc None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580/groups/fd9612958ebd4b67b99222f855b4a660/roles/048908af57db471987eb7f2ec21e0304` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.521954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ec61e-5d1d-454a-a7c4-102536495bdc None admin] RBAC: Authorizing `identity:create_grant(project_id=282dfae79c6e4e20aac9334bc958b580, group_id=fd9612958ebd4b67b99222f855b4a660, role_id=048908af57db471987eb7f2ec21e0304)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:51.524321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ec61e-5d1d-454a-a7c4-102536495bdc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:51.541286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e5bbb-ea46-4beb-a6aa-26f405dfd4b0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:51.549114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 207/410] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Aug 19 15:09:51 2022] PUT /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580/groups/fd9612958ebd4b67b99222f855b4a660/roles/048908af57db471987eb7f2ec21e0304 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:09:51.558654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afc25d-f8e3-4d26-99f2-4cd784c15a32 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.559108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afc25d-f8e3-4d26-99f2-4cd784c15a32 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.559526 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afc25d-f8e3-4d26-99f2-4cd784c15a32 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.561727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 208/411] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:51 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:51.576246 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0744ca-fabf-4bb6-bf60-e8a8e8eeb7cf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.576771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0744ca-fabf-4bb6-bf60-e8a8e8eeb7cf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.577191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0744ca-fabf-4bb6-bf60-e8a8e8eeb7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.589742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a6691-3f98-4d4d-a72f-1edb84ab3a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:51.590499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6691-3f98-4d4d-a72f-1edb84ab3a5d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.590787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6691-3f98-4d4d-a72f-1edb84ab3a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.591064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6691-3f98-4d4d-a72f-1edb84ab3a5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.606699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a6691-3f98-4d4d-a72f-1edb84ab3a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6c8a6691-3f98-4d4d-a72f-1edb84ab3a5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:51.612002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:51.612956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:51.613763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 204/412] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:09:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:51.630086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a6691-3f98-4d4d-a72f-1edb84ab3a5d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:51.640745 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-be0744ca-fabf-4bb6-bf60-e8a8e8eeb7cf None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:51.644270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be0744ca-fabf-4bb6-bf60-e8a8e8eeb7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:51.798376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 209/413] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:51.800043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38ea9b-bad4-49aa-939c-a7d132ecb64c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.800593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38ea9b-bad4-49aa-939c-a7d132ecb64c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.801071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38ea9b-bad4-49aa-939c-a7d132ecb64c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.802863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 210/414] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:51.808478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72779b65-5b91-4751-8318-d9d2c2666982 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.809057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72779b65-5b91-4751-8318-d9d2c2666982 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.809550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72779b65-5b91-4751-8318-d9d2c2666982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.828192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-72779b65-5b91-4751-8318-d9d2c2666982 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:51.829782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-72779b65-5b91-4751-8318-d9d2c2666982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:51.862302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:51.872857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:51.875460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a6691-3f98-4d4d-a72f-1edb84ab3a5d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:51.946999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ece4f-dc81-4f32-b0c2-59f66976f68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:51.947668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ece4f-dc81-4f32-b0c2-59f66976f68b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:51.947943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ece4f-dc81-4f32-b0c2-59f66976f68b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:51.949618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ece4f-dc81-4f32-b0c2-59f66976f68b None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:51.950369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ece4f-dc81-4f32-b0c2-59f66976f68b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:51.950673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ece4f-dc81-4f32-b0c2-59f66976f68b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:51.952104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ece4f-dc81-4f32-b0c2-59f66976f68b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:51.966274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 211/415] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:51.975132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 205/416] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:51.983197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72779b65-5b91-4751-8318-d9d2c2666982 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:52.116975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ece4f-dc81-4f32-b0c2-59f66976f68b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:52.157962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72779b65-5b91-4751-8318-d9d2c2666982 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:52.233946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd3cef-2e9d-4855-9420-7ca42eea9e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:52.234592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd3cef-2e9d-4855-9420-7ca42eea9e8f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:52.234873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd3cef-2e9d-4855-9420-7ca42eea9e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:52.235150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd3cef-2e9d-4855-9420-7ca42eea9e8f None admin] PATH_INFO: `/v3/groups/d2ec549a5352455d8256184a909f0a1e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:52.237628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd3cef-2e9d-4855-9420-7ca42eea9e8f None admin] RBAC: Authorizing `identity:get_group(group_id=d2ec549a5352455d8256184a909f0a1e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:52.239095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd3cef-2e9d-4855-9420-7ca42eea9e8f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:52.242002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 212/417] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:51 2022] GET /identity/v3/groups/d2ec549a5352455d8256184a909f0a1e => generated 229 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:52.249068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd3cef-2e9d-4855-9420-7ca42eea9e8f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:52.300987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:52.308740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ece4f-dc81-4f32-b0c2-59f66976f68b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:52.371641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05603620-793c-41e2-97b1-6e33a7584173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:52.372191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05603620-793c-41e2-97b1-6e33a7584173 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:52.372501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05603620-793c-41e2-97b1-6e33a7584173 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:52.372775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05603620-793c-41e2-97b1-6e33a7584173 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:52.385103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05603620-793c-41e2-97b1-6e33a7584173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-05603620-793c-41e2-97b1-6e33a7584173 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:52.390488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:52.391884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 206/418] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:52.393157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05603620-793c-41e2-97b1-6e33a7584173 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:52.420557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd3cef-2e9d-4855-9420-7ca42eea9e8f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:52.421599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:52.476861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a623d66-5327-4f00-8f0b-0b10a36b283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:52.478073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a623d66-5327-4f00-8f0b-0b10a36b283a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:52.478716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a623d66-5327-4f00-8f0b-0b10a36b283a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:52.479095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a623d66-5327-4f00-8f0b-0b10a36b283a None admin] PATH_INFO: `/v3/projects/a9a96491d392440daeaecef9eef8a152` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:52.480819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a623d66-5327-4f00-8f0b-0b10a36b283a None admin] RBAC: Authorizing `identity:get_project(project_id=a9a96491d392440daeaecef9eef8a152)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:52.483326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a623d66-5327-4f00-8f0b-0b10a36b283a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:52.487323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 213/419] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:52 2022] GET /identity/v3/projects/a9a96491d392440daeaecef9eef8a152 => generated 344 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:52.488238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a623d66-5327-4f00-8f0b-0b10a36b283a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:52.531687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05603620-793c-41e2-97b1-6e33a7584173 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:52.582461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c8ea0-90e1-47bf-8e14-5047fa6149dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:52.583759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c8ea0-90e1-47bf-8e14-5047fa6149dc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:52.584426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c8ea0-90e1-47bf-8e14-5047fa6149dc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:52.585061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c8ea0-90e1-47bf-8e14-5047fa6149dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:52.601306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c8ea0-90e1-47bf-8e14-5047fa6149dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:52.610490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2b1c8ea0-90e1-47bf-8e14-5047fa6149dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:52.611470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:52.612865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:52.614151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:52.618164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 207/420] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:52.619724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c8ea0-90e1-47bf-8e14-5047fa6149dc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:52.622515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a623d66-5327-4f00-8f0b-0b10a36b283a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:52.666786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e81ad0-7dd9-48d2-9e3b-e8dbd7b7f666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:52.667405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e81ad0-7dd9-48d2-9e3b-e8dbd7b7f666 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:52.667687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e81ad0-7dd9-48d2-9e3b-e8dbd7b7f666 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:52.667965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e81ad0-7dd9-48d2-9e3b-e8dbd7b7f666 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:52.672540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e81ad0-7dd9-48d2-9e3b-e8dbd7b7f666 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:52.672540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e81ad0-7dd9-48d2-9e3b-e8dbd7b7f666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:52.672540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e81ad0-7dd9-48d2-9e3b-e8dbd7b7f666 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:52.680738 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 214/421] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:52.681790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e81ad0-7dd9-48d2-9e3b-e8dbd7b7f666 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:52.754980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:52.789076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c8ea0-90e1-47bf-8e14-5047fa6149dc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:52.800936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e81ad0-7dd9-48d2-9e3b-e8dbd7b7f666 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:52.839573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9aa10-1874-4f2d-8870-84117e84c629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:52.840560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9aa10-1874-4f2d-8870-84117e84c629 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:52.840852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9aa10-1874-4f2d-8870-84117e84c629 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:52.841132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9aa10-1874-4f2d-8870-84117e84c629 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:52.841808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9aa10-1874-4f2d-8870-84117e84c629 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:52.842113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9aa10-1874-4f2d-8870-84117e84c629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:52.843540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9aa10-1874-4f2d-8870-84117e84c629 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:52.846668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcf075-7a25-4187-b988-8ddb270f98c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:52.848019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcf075-7a25-4187-b988-8ddb270f98c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:52.848754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcf075-7a25-4187-b988-8ddb270f98c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:52.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcf075-7a25-4187-b988-8ddb270f98c2 None admin] PATH_INFO: `/v3/projects/a9a96491d392440daeaecef9eef8a152/groups/d2ec549a5352455d8256184a909f0a1e/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:52.852861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 215/422] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:52.853612 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcf075-7a25-4187-b988-8ddb270f98c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9a96491d392440daeaecef9eef8a152, group_id=d2ec549a5352455d8256184a909f0a1e, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:52.853765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9aa10-1874-4f2d-8870-84117e84c629 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:52.857622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcf075-7a25-4187-b988-8ddb270f98c2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:52.880011 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:52.900427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 208/423] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Aug 19 15:09:52 2022] PUT /identity/v3/projects/a9a96491d392440daeaecef9eef8a152/groups/d2ec549a5352455d8256184a909f0a1e/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:09:52.900427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcf075-7a25-4187-b988-8ddb270f98c2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:53.077292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9aa10-1874-4f2d-8870-84117e84c629 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:53.115855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcf075-7a25-4187-b988-8ddb270f98c2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:53.156549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f773636-fb13-45c4-92b6-a52177314ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:53.157916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f773636-fb13-45c4-92b6-a52177314ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:53.158078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51677c84-6acb-41d2-8f4e-62b168070f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:53.158676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51677c84-6acb-41d2-8f4e-62b168070f79 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:53.158935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51677c84-6acb-41d2-8f4e-62b168070f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:53.159159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51677c84-6acb-41d2-8f4e-62b168070f79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:53.159855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f773636-fb13-45c4-92b6-a52177314ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:53.160560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f773636-fb13-45c4-92b6-a52177314ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:53.169477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:53.239121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51677c84-6acb-41d2-8f4e-62b168070f79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:53.239885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51677c84-6acb-41d2-8f4e-62b168070f79 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:53.292608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f773636-fb13-45c4-92b6-a52177314ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:53.295869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f773636-fb13-45c4-92b6-a52177314ab2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:53.331074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:53.360161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 209/424] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:53.366632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51677c84-6acb-41d2-8f4e-62b168070f79 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:53.409752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:53.456896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 216/425] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:53.458544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5257044-2eec-44b5-abd9-427eaea2ca25 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:53.459053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5257044-2eec-44b5-abd9-427eaea2ca25 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:53.459573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5257044-2eec-44b5-abd9-427eaea2ca25 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:53.461896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 217/426] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:53.470324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac172e0-7524-4399-a3ed-945be90e7ca8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:53.470655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac172e0-7524-4399-a3ed-945be90e7ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:53.471979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac172e0-7524-4399-a3ed-945be90e7ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:53.475945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51677c84-6acb-41d2-8f4e-62b168070f79 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:53.498308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aac172e0-7524-4399-a3ed-945be90e7ca8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:53.499603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aac172e0-7524-4399-a3ed-945be90e7ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:53.527680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235eb71-05ce-4089-8bee-e96d5d069e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:53.529700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235eb71-05ce-4089-8bee-e96d5d069e98 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:53.530710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235eb71-05ce-4089-8bee-e96d5d069e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:53.531585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235eb71-05ce-4089-8bee-e96d5d069e98 None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:53.534499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235eb71-05ce-4089-8bee-e96d5d069e98 None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:53.537372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235eb71-05ce-4089-8bee-e96d5d069e98 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:53.540491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 210/427] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:53 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:53.542662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235eb71-05ce-4089-8bee-e96d5d069e98 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:53.590776 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 218/428] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:53.591766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac172e0-7524-4399-a3ed-945be90e7ca8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:53.695392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:53.742569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235eb71-05ce-4089-8bee-e96d5d069e98 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:53.756774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:53.802790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac172e0-7524-4399-a3ed-945be90e7ca8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:53.816249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384102b-fb9c-42a6-84c3-f05318fa3c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:53.816926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384102b-fb9c-42a6-84c3-f05318fa3c4d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:53.817210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384102b-fb9c-42a6-84c3-f05318fa3c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:53.817475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384102b-fb9c-42a6-84c3-f05318fa3c4d None admin] PATH_INFO: `/v3/projects/c61f47242a0d40f59b342520453f190d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:53.819733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1384102b-fb9c-42a6-84c3-f05318fa3c4d None admin] RBAC: Authorizing `identity:get_project(project_id=c61f47242a0d40f59b342520453f190d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:53.821226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1384102b-fb9c-42a6-84c3-f05318fa3c4d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:53.824633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 211/429] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:53 2022] GET /identity/v3/projects/c61f47242a0d40f59b342520453f190d => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:53.828982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384102b-fb9c-42a6-84c3-f05318fa3c4d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:53.869677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637620b-4e03-4710-ad67-6f3dc2ca88fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:53.871660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637620b-4e03-4710-ad67-6f3dc2ca88fb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:53.872440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637620b-4e03-4710-ad67-6f3dc2ca88fb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:53.873169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637620b-4e03-4710-ad67-6f3dc2ca88fb None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:53.875274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637620b-4e03-4710-ad67-6f3dc2ca88fb None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:53.878048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637620b-4e03-4710-ad67-6f3dc2ca88fb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:53.881537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 219/430] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:53 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:53.883103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637620b-4e03-4710-ad67-6f3dc2ca88fb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:53.945243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384102b-fb9c-42a6-84c3-f05318fa3c4d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:53.982533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e886b7-2cc5-40e9-9382-964f0cb1d28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:53.983176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e886b7-2cc5-40e9-9382-964f0cb1d28a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:53.983453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e886b7-2cc5-40e9-9382-964f0cb1d28a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:53.983728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e886b7-2cc5-40e9-9382-964f0cb1d28a None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:53.985176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e886b7-2cc5-40e9-9382-964f0cb1d28a None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:53.986639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e886b7-2cc5-40e9-9382-964f0cb1d28a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:53.989072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 212/431] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:53 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:53.989961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e886b7-2cc5-40e9-9382-964f0cb1d28a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:54.043632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:54.070720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637620b-4e03-4710-ad67-6f3dc2ca88fb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:54.089848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:54.116909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e886b7-2cc5-40e9-9382-964f0cb1d28a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:54.140927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bb9c6-3b0f-4c95-8ec5-5f4f60896035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:54.140927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bb9c6-3b0f-4c95-8ec5-5f4f60896035 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:54.140927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bb9c6-3b0f-4c95-8ec5-5f4f60896035 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:54.140927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bb9c6-3b0f-4c95-8ec5-5f4f60896035 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:54.168108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bb9c6-3b0f-4c95-8ec5-5f4f60896035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:54.173073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf351-32c3-4860-ade0-7c3b91caa076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:54.173688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf351-32c3-4860-ade0-7c3b91caa076 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:54.174558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf351-32c3-4860-ade0-7c3b91caa076 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:54.174558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf351-32c3-4860-ade0-7c3b91caa076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:54.174973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf351-32c3-4860-ade0-7c3b91caa076 None admin] RBAC: Adding query filter params (group.id=fd9612958ebd4b67b99222f855b4a660, role.id=a7f0f98b4677401690c354299b049892, scope.project.id=c61f47242a0d40f59b342520453f190d) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:54.175280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf351-32c3-4860-ade0-7c3b91caa076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-611bb9c6-3b0f-4c95-8ec5-5f4f60896035 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:54.176800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:54.177668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 15:09:54.178569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:54.179170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf351-32c3-4860-ade0-7c3b91caa076 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:54.183345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 220/432] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:54.184429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bb9c6-3b0f-4c95-8ec5-5f4f60896035 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:54.186829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 213/433] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:53 2022] GET /identity/v3/role_assignments?group.id=fd9612958ebd4b67b99222f855b4a660&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=a7f0f98b4677401690c354299b049892 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:54.187512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf351-32c3-4860-ade0-7c3b91caa076 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:54.372666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bb9c6-3b0f-4c95-8ec5-5f4f60896035 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:54.443577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf351-32c3-4860-ade0-7c3b91caa076 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:54.451733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e629cfb-c996-4776-ba60-2069956c5966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:54.452829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e629cfb-c996-4776-ba60-2069956c5966 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:54.453310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e629cfb-c996-4776-ba60-2069956c5966 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:54.453790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e629cfb-c996-4776-ba60-2069956c5966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:54.455126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e629cfb-c996-4776-ba60-2069956c5966 None admin] RBAC: Adding query filter params (group.id=fd9612958ebd4b67b99222f855b4a660, role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=c61f47242a0d40f59b342520453f190d) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:54.455668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e629cfb-c996-4776-ba60-2069956c5966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:54.457769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:54.462169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e629cfb-c996-4776-ba60-2069956c5966 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:54.476550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 221/434] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:54 2022] GET /identity/v3/role_assignments?group.id=fd9612958ebd4b67b99222f855b4a660&scope.project.id=c61f47242a0d40f59b342520453f190d&role.id=048908af57db471987eb7f2ec21e0304 => generated 611 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:54.481208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f295d6-7784-4968-b2bd-d3e19a2a0da9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:54.483532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f295d6-7784-4968-b2bd-d3e19a2a0da9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:54.485504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f295d6-7784-4968-b2bd-d3e19a2a0da9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:54.497478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 222/435] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:54 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:54.502921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa63e3f-ba85-434a-8594-e59064390c70 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:54.504431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa63e3f-ba85-434a-8594-e59064390c70 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:54.505822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa63e3f-ba85-434a-8594-e59064390c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:54.518603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20243a-cbb6-49f2-a5aa-1f3af0e5b7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:54.520010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20243a-cbb6-49f2-a5aa-1f3af0e5b7bd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:54.520296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20243a-cbb6-49f2-a5aa-1f3af0e5b7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:54.521472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20243a-cbb6-49f2-a5aa-1f3af0e5b7bd None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:54.522187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20243a-cbb6-49f2-a5aa-1f3af0e5b7bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:54.522494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20243a-cbb6-49f2-a5aa-1f3af0e5b7bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:54.527957 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20243a-cbb6-49f2-a5aa-1f3af0e5b7bd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:54.553168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 214/436] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:09:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:54.554201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20243a-cbb6-49f2-a5aa-1f3af0e5b7bd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:54.560062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1aa63e3f-ba85-434a-8594-e59064390c70 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:54.567377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1aa63e3f-ba85-434a-8594-e59064390c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:54.666034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:54.754096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 223/437] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:09:54.786155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20243a-cbb6-49f2-a5aa-1f3af0e5b7bd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:54.829536 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a8ae9-5938-42c7-a089-a861a3319766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:54.830243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a8ae9-5938-42c7-a089-a861a3319766 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:54.830529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a8ae9-5938-42c7-a089-a861a3319766 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:54.830802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a8ae9-5938-42c7-a089-a861a3319766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:54.874229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:54.915523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a8ae9-5938-42c7-a089-a861a3319766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:54.917603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a8ae9-5938-42c7-a089-a861a3319766 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:55.036489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 215/438] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:09:55.038753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa63e3f-ba85-434a-8594-e59064390c70 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:55.147796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa63e3f-ba85-434a-8594-e59064390c70 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:55.184494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabc0ed-99a3-437c-84a5-6580dc5e95e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:55.185441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabc0ed-99a3-437c-84a5-6580dc5e95e1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:55.185935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabc0ed-99a3-437c-84a5-6580dc5e95e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:55.186415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabc0ed-99a3-437c-84a5-6580dc5e95e1 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:55.187710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabc0ed-99a3-437c-84a5-6580dc5e95e1 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:55.189377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabc0ed-99a3-437c-84a5-6580dc5e95e1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:55.191268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 224/439] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:55 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:55.196895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a8ae9-5938-42c7-a089-a861a3319766 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:55.224823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabc0ed-99a3-437c-84a5-6580dc5e95e1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:55.268143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:55.276344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:55.303358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a8ae9-5938-42c7-a089-a861a3319766 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:55.339814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2315173-da84-4296-9bc3-7e668fd2b5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:55.340467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2315173-da84-4296-9bc3-7e668fd2b5ce None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:55.340749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2315173-da84-4296-9bc3-7e668fd2b5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:55.341021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2315173-da84-4296-9bc3-7e668fd2b5ce None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:55.342174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2315173-da84-4296-9bc3-7e668fd2b5ce None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:55.343624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2315173-da84-4296-9bc3-7e668fd2b5ce None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:55.345216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 216/440] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:55 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:55.346318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105d5dc-cc58-4b47-a48e-0844af9b714b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:55.346656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105d5dc-cc58-4b47-a48e-0844af9b714b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:55.346942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105d5dc-cc58-4b47-a48e-0844af9b714b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:55.348123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 217/441] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:55.353202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105d5dc-cc58-4b47-a48e-0844af9b714b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:55.358856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabc0ed-99a3-437c-84a5-6580dc5e95e1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:55.403540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4c1e4-442e-457b-a275-53e7c7078ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:55.404568 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c1e4-442e-457b-a275-53e7c7078ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:55.405026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c1e4-442e-457b-a275-53e7c7078ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:55.405471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4c1e4-442e-457b-a275-53e7c7078ed0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:55.421045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4c1e4-442e-457b-a275-53e7c7078ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-22c4c1e4-442e-457b-a275-53e7c7078ed0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:55.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:55.430641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:09:55.431426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 225/442] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:09:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:09:55.433386 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd3db2-472b-4458-a976-38259f379fc8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:55.433866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd3db2-472b-4458-a976-38259f379fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:55.434337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd3db2-472b-4458-a976-38259f379fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:55.455299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-08cd3db2-472b-4458-a976-38259f379fc8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:55.456424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-08cd3db2-472b-4458-a976-38259f379fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:55.478628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105d5dc-cc58-4b47-a48e-0844af9b714b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:55.514746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab1763-b511-4ab8-a789-4451c512931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:55.515351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab1763-b511-4ab8-a789-4451c512931c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:55.515635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab1763-b511-4ab8-a789-4451c512931c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:55.515906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab1763-b511-4ab8-a789-4451c512931c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:55.516662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab1763-b511-4ab8-a789-4451c512931c None admin] RBAC: Adding query filter params (group.id=fd9612958ebd4b67b99222f855b4a660, role.id=a7f0f98b4677401690c354299b049892, scope.project.id=282dfae79c6e4e20aac9334bc958b580) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:55.516977 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab1763-b511-4ab8-a789-4451c512931c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:55.518454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab1763-b511-4ab8-a789-4451c512931c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:55.521375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:55.525213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 218/443] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:55 2022] GET /identity/v3/role_assignments?group.id=fd9612958ebd4b67b99222f855b4a660&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=a7f0f98b4677401690c354299b049892 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:55.526778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 226/444] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:09:55.527716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab1763-b511-4ab8-a789-4451c512931c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:55.651775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab1763-b511-4ab8-a789-4451c512931c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:55.706076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c86bd-f610-4804-b929-f9a4ef671902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:55.707389 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c86bd-f610-4804-b929-f9a4ef671902 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:55.708050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c86bd-f610-4804-b929-f9a4ef671902 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:55.708732 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c86bd-f610-4804-b929-f9a4ef671902 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:55.710086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c86bd-f610-4804-b929-f9a4ef671902 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:55.710810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c86bd-f610-4804-b929-f9a4ef671902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:55.715036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c86bd-f610-4804-b929-f9a4ef671902 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:55.727682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 219/445] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:09:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:55.735769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd3db2-472b-4458-a976-38259f379fc8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:55.774635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:55.797104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c86bd-f610-4804-b929-f9a4ef671902 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:55.865176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd3db2-472b-4458-a976-38259f379fc8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:55.895385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:55.924750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897578-cd95-4b6a-b294-735b4c1f72e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:55.925790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897578-cd95-4b6a-b294-735b4c1f72e2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:55.926011 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897578-cd95-4b6a-b294-735b4c1f72e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:55.926455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897578-cd95-4b6a-b294-735b4c1f72e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:55.956297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c86bd-f610-4804-b929-f9a4ef671902 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:55.995915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9de7b1-6e55-4e76-a1f8-4c14784c1930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:55.996755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9de7b1-6e55-4e76-a1f8-4c14784c1930 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:55.997033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9de7b1-6e55-4e76-a1f8-4c14784c1930 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:55.997301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9de7b1-6e55-4e76-a1f8-4c14784c1930 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:56.007694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9de7b1-6e55-4e76-a1f8-4c14784c1930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:09:56.008180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61897578-cd95-4b6a-b294-735b4c1f72e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:56.009807 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61897578-cd95-4b6a-b294-735b4c1f72e2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2c9de7b1-6e55-4e76-a1f8-4c14784c1930 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:09:56.014633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:09:56.015560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:09:56.016544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 220/446] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:09:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:09:56.021688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9de7b1-6e55-4e76-a1f8-4c14784c1930 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:56.042770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:56.136674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9de7b1-6e55-4e76-a1f8-4c14784c1930 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:56.139173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 227/447] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:09:56.145211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897578-cd95-4b6a-b294-735b4c1f72e2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:56.162144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:56.175558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb3e76-d4b4-4768-8836-f4ee0af74654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:56.176168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb3e76-d4b4-4768-8836-f4ee0af74654 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:56.176475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb3e76-d4b4-4768-8836-f4ee0af74654 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:56.176750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb3e76-d4b4-4768-8836-f4ee0af74654 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:56.177425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb3e76-d4b4-4768-8836-f4ee0af74654 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:56.177729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb3e76-d4b4-4768-8836-f4ee0af74654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:56.179130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb3e76-d4b4-4768-8836-f4ee0af74654 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:56.187510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 221/448] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:09:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:56.193167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb3e76-d4b4-4768-8836-f4ee0af74654 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:56.259382 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897578-cd95-4b6a-b294-735b4c1f72e2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:56.297220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:56.309002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca1a4e-a437-45cb-b8c0-4dc92b7eab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:56.309654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca1a4e-a437-45cb-b8c0-4dc92b7eab47 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:56.309938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca1a4e-a437-45cb-b8c0-4dc92b7eab47 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:56.310212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca1a4e-a437-45cb-b8c0-4dc92b7eab47 None admin] PATH_INFO: `/v3/projects/282dfae79c6e4e20aac9334bc958b580` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:56.311386 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca1a4e-a437-45cb-b8c0-4dc92b7eab47 None admin] RBAC: Authorizing `identity:get_project(project_id=282dfae79c6e4e20aac9334bc958b580)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:56.312888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca1a4e-a437-45cb-b8c0-4dc92b7eab47 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:56.314532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 228/449] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:56 2022] GET /identity/v3/projects/282dfae79c6e4e20aac9334bc958b580 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:56.323041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca1a4e-a437-45cb-b8c0-4dc92b7eab47 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:56.323570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb3e76-d4b4-4768-8836-f4ee0af74654 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:56.360888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7f67b-d418-4bbb-a9cc-e2978369dfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:56.361501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7f67b-d418-4bbb-a9cc-e2978369dfe0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:56.361784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7f67b-d418-4bbb-a9cc-e2978369dfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:56.362058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7f67b-d418-4bbb-a9cc-e2978369dfe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:56.420342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:56.438832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca1a4e-a437-45cb-b8c0-4dc92b7eab47 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:56.439470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7f67b-d418-4bbb-a9cc-e2978369dfe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:56.440980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7f67b-d418-4bbb-a9cc-e2978369dfe0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:56.477687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aa319-74f6-46c7-8306-485f218ba867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:56.478320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aa319-74f6-46c7-8306-485f218ba867 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:56.478610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aa319-74f6-46c7-8306-485f218ba867 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:56.478884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aa319-74f6-46c7-8306-485f218ba867 None admin] PATH_INFO: `/v3/groups/fd9612958ebd4b67b99222f855b4a660` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:56.480669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aa319-74f6-46c7-8306-485f218ba867 None admin] RBAC: Authorizing `identity:get_group(group_id=fd9612958ebd4b67b99222f855b4a660)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:56.482136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3aa319-74f6-46c7-8306-485f218ba867 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:56.484034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 229/450] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:56 2022] GET /identity/v3/groups/fd9612958ebd4b67b99222f855b4a660 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:56.489717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aa319-74f6-46c7-8306-485f218ba867 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:56.554242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:56.555052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 222/451] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:09:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:09:56.561608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7f67b-d418-4bbb-a9cc-e2978369dfe0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:56.597336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3aa319-74f6-46c7-8306-485f218ba867 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:56.636479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4787e4b-bd01-4655-8050-cb3beab36228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:56.637089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4787e4b-bd01-4655-8050-cb3beab36228 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:56.637370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4787e4b-bd01-4655-8050-cb3beab36228 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:56.637643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4787e4b-bd01-4655-8050-cb3beab36228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:56.638372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4787e4b-bd01-4655-8050-cb3beab36228 None admin] RBAC: Adding query filter params (group.id=fd9612958ebd4b67b99222f855b4a660, role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=282dfae79c6e4e20aac9334bc958b580) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:56.638704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4787e4b-bd01-4655-8050-cb3beab36228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:56.640127 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4787e4b-bd01-4655-8050-cb3beab36228 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:56.645920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 230/452] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:56 2022] GET /identity/v3/role_assignments?group.id=fd9612958ebd4b67b99222f855b4a660&scope.project.id=282dfae79c6e4e20aac9334bc958b580&role.id=048908af57db471987eb7f2ec21e0304 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:56.656514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:56.676420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7f67b-d418-4bbb-a9cc-e2978369dfe0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:56.715515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1042aeb-e63b-4757-ad4a-ae07525de0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:56.716167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1042aeb-e63b-4757-ad4a-ae07525de0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:56.716481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1042aeb-e63b-4757-ad4a-ae07525de0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:56.716752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1042aeb-e63b-4757-ad4a-ae07525de0d4 None admin] PATH_INFO: `/v3/projects/a9a96491d392440daeaecef9eef8a152` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:56.717951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1042aeb-e63b-4757-ad4a-ae07525de0d4 None admin] RBAC: Authorizing `identity:get_project(project_id=a9a96491d392440daeaecef9eef8a152)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:56.719465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1042aeb-e63b-4757-ad4a-ae07525de0d4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:56.721194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 223/453] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:09:56 2022] GET /identity/v3/projects/a9a96491d392440daeaecef9eef8a152 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:09:56.727108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4787e4b-bd01-4655-8050-cb3beab36228 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:56.839464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4787e4b-bd01-4655-8050-cb3beab36228 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:56.867606 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:56.878838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29279c78-e4eb-40ea-af5b-0eb2c2709889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:56.879777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29279c78-e4eb-40ea-af5b-0eb2c2709889 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:56.880106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29279c78-e4eb-40ea-af5b-0eb2c2709889 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:56.880497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29279c78-e4eb-40ea-af5b-0eb2c2709889 None admin] PATH_INFO: `/v3/groups/d2ec549a5352455d8256184a909f0a1e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:56.881997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29279c78-e4eb-40ea-af5b-0eb2c2709889 None admin] RBAC: Authorizing `identity:get_group(group_id=d2ec549a5352455d8256184a909f0a1e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:56.883580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29279c78-e4eb-40ea-af5b-0eb2c2709889 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:56.885218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 231/454] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Aug 19 15:09:56 2022] GET /identity/v3/groups/d2ec549a5352455d8256184a909f0a1e => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:56.891719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1042aeb-e63b-4757-ad4a-ae07525de0d4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:09:57.004846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1042aeb-e63b-4757-ad4a-ae07525de0d4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:09:57.039685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d1f48-1609-4f6b-a08a-0fe1c494ef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:09:57.040280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1f48-1609-4f6b-a08a-0fe1c494ef1e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:57.040597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1f48-1609-4f6b-a08a-0fe1c494ef1e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:57.040875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d1f48-1609-4f6b-a08a-0fe1c494ef1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:57.041608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d1f48-1609-4f6b-a08a-0fe1c494ef1e None admin] RBAC: Adding query filter params (group.id=d2ec549a5352455d8256184a909f0a1e, role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=a9a96491d392440daeaecef9eef8a152) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:09:57.042463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d1f48-1609-4f6b-a08a-0fe1c494ef1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:09:57.044000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d1f48-1609-4f6b-a08a-0fe1c494ef1e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:09:57.049946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 224/455] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Aug 19 15:09:56 2022] GET /identity/v3/role_assignments?group.id=d2ec549a5352455d8256184a909f0a1e&scope.project.id=a9a96491d392440daeaecef9eef8a152&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:09:59.371326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff31295-e3c9-4e18-bc60-7de15e2f2565 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:59.371876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff31295-e3c9-4e18-bc60-7de15e2f2565 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:59.371876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff31295-e3c9-4e18-bc60-7de15e2f2565 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:59.373236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 232/456] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:09:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:09:59.378527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a05cf-c504-4024-a1f5-703a6588b222 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:09:59.379303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a05cf-c504-4024-a1f5-703a6588b222 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:09:59.379733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a05cf-c504-4024-a1f5-703a6588b222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:09:59.409415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fe8a05cf-c504-4024-a1f5-703a6588b222 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:09:59.410851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fe8a05cf-c504-4024-a1f5-703a6588b222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:09:59.426083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:09:59.504630 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 225/457] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:09:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:00.007132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff31295-e3c9-4e18-bc60-7de15e2f2565 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:00.041038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c6f76-2f4d-4402-9a2b-224eef391b81 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:00.041704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c6f76-2f4d-4402-9a2b-224eef391b81 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:00.042859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c6f76-2f4d-4402-9a2b-224eef391b81 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:00.044701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 226/458] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:00.051517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c5e28-8066-4b94-9df4-c0b14dfc5af0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:00.051799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c5e28-8066-4b94-9df4-c0b14dfc5af0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:00.052068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c5e28-8066-4b94-9df4-c0b14dfc5af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:00.077660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-220c5e28-8066-4b94-9df4-c0b14dfc5af0 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:00.081803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-220c5e28-8066-4b94-9df4-c0b14dfc5af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:00.159767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff31295-e3c9-4e18-bc60-7de15e2f2565 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:00.185911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 227/459] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:10:00.187871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903cc81-6ec0-4c50-a821-7b39c984126d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:00.188474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903cc81-6ec0-4c50-a821-7b39c984126d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:00.188920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903cc81-6ec0-4c50-a821-7b39c984126d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:00.190987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 228/460] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:00.197776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52107ff6-f5bb-4980-a505-bda2c48f2927 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:00.199954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52107ff6-f5bb-4980-a505-bda2c48f2927 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:00.200357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52107ff6-f5bb-4980-a505-bda2c48f2927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:00.220159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-52107ff6-f5bb-4980-a505-bda2c48f2927 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:00.221657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-52107ff6-f5bb-4980-a505-bda2c48f2927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:00.241687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c9d14-cdb7-4842-a38a-6576c7ca4d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:00.242345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c9d14-cdb7-4842-a38a-6576c7ca4d06 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:00.242626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c9d14-cdb7-4842-a38a-6576c7ca4d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:00.242896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c9d14-cdb7-4842-a38a-6576c7ca4d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:00.270034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c9d14-cdb7-4842-a38a-6576c7ca4d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:00.272893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c9d14-cdb7-4842-a38a-6576c7ca4d06 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7d6c9d14-cdb7-4842-a38a-6576c7ca4d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:00.302966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:00.304141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:00.305205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:00.305205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:00.305205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:00.311254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 233/461] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:00.315842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c9d14-cdb7-4842-a38a-6576c7ca4d06 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:00.337406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 229/462] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:00.376303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:00.437099 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c9d14-cdb7-4842-a38a-6576c7ca4d06 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:00.474453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50978-0787-4c73-894f-25ada949dc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:00.475400 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50978-0787-4c73-894f-25ada949dc02 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:00.475819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50978-0787-4c73-894f-25ada949dc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:00.476225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50978-0787-4c73-894f-25ada949dc02 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:00.477026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50978-0787-4c73-894f-25ada949dc02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:00.477462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50978-0787-4c73-894f-25ada949dc02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:00.479070 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50978-0787-4c73-894f-25ada949dc02 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:00.493254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 234/463] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:00.499558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52107ff6-f5bb-4980-a505-bda2c48f2927 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:00.508841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:00.641346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52107ff6-f5bb-4980-a505-bda2c48f2927 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:00.659651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b4457-0ef1-4983-8826-86b75dbe9a14 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:00.660150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b4457-0ef1-4983-8826-86b75dbe9a14 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:00.660581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b4457-0ef1-4983-8826-86b75dbe9a14 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:00.662322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 235/464] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:00.672585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e5e34-c857-4167-8a60-bc0e1c2f75b6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:00.673177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e5e34-c857-4167-8a60-bc0e1c2f75b6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:00.673562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e5e34-c857-4167-8a60-bc0e1c2f75b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:00.710846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e62e5e34-c857-4167-8a60-bc0e1c2f75b6 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:00.712052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e62e5e34-c857-4167-8a60-bc0e1c2f75b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:00.715434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86318e7d-726b-465c-93a9-f1c8708b46c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:00.716044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86318e7d-726b-465c-93a9-f1c8708b46c2 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:00.716319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86318e7d-726b-465c-93a9-f1c8708b46c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:00.716624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86318e7d-726b-465c-93a9-f1c8708b46c2 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:00.717265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86318e7d-726b-465c-93a9-f1c8708b46c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:00.718939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86318e7d-726b-465c-93a9-f1c8708b46c2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:00.758910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 230/465] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:10:00 2022] POST /identity/v3/users => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:10:00.760374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86318e7d-726b-465c-93a9-f1c8708b46c2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:00.867360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 236/466] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:00.868440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e5e34-c857-4167-8a60-bc0e1c2f75b6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:00.910762 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:00.918374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:00.972430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86318e7d-726b-465c-93a9-f1c8708b46c2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:01.015185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64a371-6deb-4986-8b0b-68e1662e8535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:01.015917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64a371-6deb-4986-8b0b-68e1662e8535 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.016271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64a371-6deb-4986-8b0b-68e1662e8535 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.016666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64a371-6deb-4986-8b0b-68e1662e8535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.018894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e5e34-c857-4167-8a60-bc0e1c2f75b6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:01.030460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64a371-6deb-4986-8b0b-68e1662e8535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:01.032136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64a371-6deb-4986-8b0b-68e1662e8535 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ad64a371-6deb-4986-8b0b-68e1662e8535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:01.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:01.049998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:01.051092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:01.051092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:01.051092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:01.051092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 231/467] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:01.059942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64a371-6deb-4986-8b0b-68e1662e8535 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:01.089348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb5930-d658-4952-acdb-dd4740cb971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:01.090315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb5930-d658-4952-acdb-dd4740cb971f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.090744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb5930-d658-4952-acdb-dd4740cb971f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.091166 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb5930-d658-4952-acdb-dd4740cb971f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.108543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb5930-d658-4952-acdb-dd4740cb971f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:01.110960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb5930-d658-4952-acdb-dd4740cb971f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7afb5930-d658-4952-acdb-dd4740cb971f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:01.136554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:01.138652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:01.139685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:01.139685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:01.139685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:01.139685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 237/468] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:01.145606 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb5930-d658-4952-acdb-dd4740cb971f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:01.181943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64a371-6deb-4986-8b0b-68e1662e8535 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:01.198540 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:01.220173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc6d1b-495d-43f2-98f9-c3b8fdd8659e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:01.220860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc6d1b-495d-43f2-98f9-c3b8fdd8659e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.221148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc6d1b-495d-43f2-98f9-c3b8fdd8659e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.221426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc6d1b-495d-43f2-98f9-c3b8fdd8659e None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.222091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc6d1b-495d-43f2-98f9-c3b8fdd8659e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:01.222457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc6d1b-495d-43f2-98f9-c3b8fdd8659e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:01.224174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc6d1b-495d-43f2-98f9-c3b8fdd8659e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:01.243085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 232/469] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:01.252145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c45c1f-1382-4470-a24e-2f50783d7995 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.252616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c45c1f-1382-4470-a24e-2f50783d7995 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.253027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c45c1f-1382-4470-a24e-2f50783d7995 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.255535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 233/470] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:01.256700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c45c1f-1382-4470-a24e-2f50783d7995 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:01.284146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:01.298789 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb5930-d658-4952-acdb-dd4740cb971f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:01.337585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b7863-e6fc-4c97-b57d-281dd4fd5eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:01.338454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b7863-e6fc-4c97-b57d-281dd4fd5eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.338942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b7863-e6fc-4c97-b57d-281dd4fd5eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.339363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b7863-e6fc-4c97-b57d-281dd4fd5eb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.340238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b7863-e6fc-4c97-b57d-281dd4fd5eb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:01.340837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b7863-e6fc-4c97-b57d-281dd4fd5eb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:01.342384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b7863-e6fc-4c97-b57d-281dd4fd5eb3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:01.360336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 238/471] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:01.361976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8914a-612f-4dcc-b7c4-6a45b5c01f8f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.362408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8914a-612f-4dcc-b7c4-6a45b5c01f8f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.362833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8914a-612f-4dcc-b7c4-6a45b5c01f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.381844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-38b8914a-612f-4dcc-b7c4-6a45b5c01f8f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:01.383757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-38b8914a-612f-4dcc-b7c4-6a45b5c01f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:01.423567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c45c1f-1382-4470-a24e-2f50783d7995 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:01.457577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 239/472] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:01.458728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8914a-612f-4dcc-b7c4-6a45b5c01f8f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:01.477813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd490da-b9b7-4d21-8d78-9bae4aa722a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:01.478736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd490da-b9b7-4d21-8d78-9bae4aa722a4 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.479820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd490da-b9b7-4d21-8d78-9bae4aa722a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.480367 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd490da-b9b7-4d21-8d78-9bae4aa722a4 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.481080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd490da-b9b7-4d21-8d78-9bae4aa722a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:01.482535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd490da-b9b7-4d21-8d78-9bae4aa722a4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:01.509354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 234/473] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:10:01 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:01.510599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd490da-b9b7-4d21-8d78-9bae4aa722a4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:01.553474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:01.610652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:01.644999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd490da-b9b7-4d21-8d78-9bae4aa722a4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:01.681983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8914a-612f-4dcc-b7c4-6a45b5c01f8f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:01.688521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c06c8-455a-4ed4-b28a-c49857a211ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:01.689546 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c06c8-455a-4ed4-b28a-c49857a211ab None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.691099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c06c8-455a-4ed4-b28a-c49857a211ab None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.691602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c06c8-455a-4ed4-b28a-c49857a211ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.704995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c06c8-455a-4ed4-b28a-c49857a211ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:01.706872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c06c8-455a-4ed4-b28a-c49857a211ab None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-eb4c06c8-455a-4ed4-b28a-c49857a211ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:01.725078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:01.726028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:01.726887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:01.726887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:01.726887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:01.729850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 235/474] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:01.742519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c06c8-455a-4ed4-b28a-c49857a211ab None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:01.779908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd986b-2603-4d0c-9847-6458da6b1160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:01.780914 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd986b-2603-4d0c-9847-6458da6b1160 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.781499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd986b-2603-4d0c-9847-6458da6b1160 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.781909 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd986b-2603-4d0c-9847-6458da6b1160 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.783098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd986b-2603-4d0c-9847-6458da6b1160 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:01.785622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd986b-2603-4d0c-9847-6458da6b1160 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:01.821092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 240/475] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:10:01 2022] POST /identity/v3/users => generated 249 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:01.822488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd986b-2603-4d0c-9847-6458da6b1160 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:01.851371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:01.877048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c06c8-455a-4ed4-b28a-c49857a211ab None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:01.924429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ddaf3-4df3-4e06-9e38-b3299b7d59b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:01.925487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ddaf3-4df3-4e06-9e38-b3299b7d59b6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:01.926710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ddaf3-4df3-4e06-9e38-b3299b7d59b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:01.927255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ddaf3-4df3-4e06-9e38-b3299b7d59b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:01.928475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ddaf3-4df3-4e06-9e38-b3299b7d59b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:01.929016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ddaf3-4df3-4e06-9e38-b3299b7d59b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:01.931037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ddaf3-4df3-4e06-9e38-b3299b7d59b6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:01.950200 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 236/476] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:01.957850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ddaf3-4df3-4e06-9e38-b3299b7d59b6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:02.022726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd986b-2603-4d0c-9847-6458da6b1160 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:02.087514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:02.088701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:02.089228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:02.089853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:02.090968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:02.093588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:02.095590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ddaf3-4df3-4e06-9e38-b3299b7d59b6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:02.099304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:02.108333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.sql.core [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 19 15:10:02.108333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 19 15:10:02.108333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [parameters: {'id': '71192495139741dfae95ed1d1b62f7a6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 19 15:10:02.108333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 19 15:10:02.139619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa9a42-8533-465d-be0b-6437abec3719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:02.140572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa9a42-8533-465d-be0b-6437abec3719 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Aug 19 15:10:02.140832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return ref.to_dict() Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 15:10:02.141849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application yield resource Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.rollback() Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.force_reraise() Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise self.value Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application session.commit() Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 19 15:10:02.143008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._prepare_impl() Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.flush() Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._flush(objects) Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application compat.raise_( Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application flush_context.execute() Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rec.execute(self) Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 19 15:10:02.144072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 15:10:02.152533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 15:10:02.154600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [parameters: {'id': '71192495139741dfae95ed1d1b62f7a6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:02.156649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 19 15:10:02.158737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:02.160894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 241/477] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Aug 19 15:10:01 2022] POST /identity/v3/roles => generated 144 bytes in 321 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:02.161039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa9a42-8533-465d-be0b-6437abec3719 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:02.161039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa9a42-8533-465d-be0b-6437abec3719 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:02.161039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa9a42-8533-465d-be0b-6437abec3719 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:02.161039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa9a42-8533-465d-be0b-6437abec3719 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:02.161583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:02.205807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:02.219304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 237/478] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:10:01 2022] POST /identity/v3/users => generated 248 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:10:02.335393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5d301-0399-4b96-9861-f2f18859fdf6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:02.393218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d68ff1-4af4-4d1a-a960-1ea5e320b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:02.394182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d68ff1-4af4-4d1a-a960-1ea5e320b1db None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:02.394540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d68ff1-4af4-4d1a-a960-1ea5e320b1db None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:02.394949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d68ff1-4af4-4d1a-a960-1ea5e320b1db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:02.412976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d68ff1-4af4-4d1a-a960-1ea5e320b1db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-17d68ff1-4af4-4d1a-a960-1ea5e320b1db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:02.421999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:02.423768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:02.425338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 242/479] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:02.438118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa9a42-8533-465d-be0b-6437abec3719 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:02.630314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa9a42-8533-465d-be0b-6437abec3719 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:02.696774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c034034-232c-4a55-a6df-10681d27efcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:02.697748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c034034-232c-4a55-a6df-10681d27efcd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:02.698235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c034034-232c-4a55-a6df-10681d27efcd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:02.699020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c034034-232c-4a55-a6df-10681d27efcd None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:02.700986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c034034-232c-4a55-a6df-10681d27efcd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:02.701518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c034034-232c-4a55-a6df-10681d27efcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:02.705023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c034034-232c-4a55-a6df-10681d27efcd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:02.720609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 238/480] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:03.121455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0a181-f3ce-4844-8598-38ad09239094 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:03.121857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0a181-f3ce-4844-8598-38ad09239094 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:03.122049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0a181-f3ce-4844-8598-38ad09239094 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:03.123575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 243/481] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:03.129228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbedb9-b9f9-4d7b-ab9e-d0dea6c11b15 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:03.129759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbedb9-b9f9-4d7b-ab9e-d0dea6c11b15 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:03.130257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbedb9-b9f9-4d7b-ab9e-d0dea6c11b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:03.149983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1fdbedb9-b9f9-4d7b-ab9e-d0dea6c11b15 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:03.151945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1fdbedb9-b9f9-4d7b-ab9e-d0dea6c11b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:03.214246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:03.232367 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 239/482] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:03.553740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0a181-f3ce-4844-8598-38ad09239094 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:03.618453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:03.664795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0a181-f3ce-4844-8598-38ad09239094 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:03.712131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347449-3f5e-4fc3-99f7-962f5ca25c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:03.712794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347449-3f5e-4fc3-99f7-962f5ca25c15 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:03.713078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347449-3f5e-4fc3-99f7-962f5ca25c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:03.713358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347449-3f5e-4fc3-99f7-962f5ca25c15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:03.725642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1347449-3f5e-4fc3-99f7-962f5ca25c15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f1347449-3f5e-4fc3-99f7-962f5ca25c15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:03.731136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:03.732050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:03.732865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:03.736525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 244/483] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:03.738569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbedb9-b9f9-4d7b-ab9e-d0dea6c11b15 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:03.866442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbedb9-b9f9-4d7b-ab9e-d0dea6c11b15 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:03.916479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4cf1e-abee-4c6e-b60c-9c283a3050c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:03.917653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4cf1e-abee-4c6e-b60c-9c283a3050c0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:03.917935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4cf1e-abee-4c6e-b60c-9c283a3050c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:03.918212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4cf1e-abee-4c6e-b60c-9c283a3050c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:03.918895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4cf1e-abee-4c6e-b60c-9c283a3050c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:03.919200 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4cf1e-abee-4c6e-b60c-9c283a3050c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:03.920621 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4cf1e-abee-4c6e-b60c-9c283a3050c0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:03.931959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 240/484] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:03.937613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347449-3f5e-4fc3-99f7-962f5ca25c15 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:04.058843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347449-3f5e-4fc3-99f7-962f5ca25c15 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:04.077413 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:04.095598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89d41e-f430-4066-b3be-6d9f835f9472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:04.096230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89d41e-f430-4066-b3be-6d9f835f9472 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:04.096542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89d41e-f430-4066-b3be-6d9f835f9472 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:04.096815 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89d41e-f430-4066-b3be-6d9f835f9472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:04.107994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89d41e-f430-4066-b3be-6d9f835f9472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:04.109626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89d41e-f430-4066-b3be-6d9f835f9472 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6a89d41e-f430-4066-b3be-6d9f835f9472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:04.124516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:04.125420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:04.126329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:04.126329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:04.126329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:04.129967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 245/485] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:04.131762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4cf1e-abee-4c6e-b60c-9c283a3050c0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:04.207448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:04.265249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4cf1e-abee-4c6e-b60c-9c283a3050c0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:04.306606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932284-0215-4f38-8e1f-ea16304cc98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:04.307288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932284-0215-4f38-8e1f-ea16304cc98c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:04.308231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932284-0215-4f38-8e1f-ea16304cc98c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:04.308757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932284-0215-4f38-8e1f-ea16304cc98c None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:04.309444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932284-0215-4f38-8e1f-ea16304cc98c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:04.309745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932284-0215-4f38-8e1f-ea16304cc98c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:04.311140 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932284-0215-4f38-8e1f-ea16304cc98c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:04.328115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 241/486] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:04.333637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89d41e-f430-4066-b3be-6d9f835f9472 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:04.456469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89d41e-f430-4066-b3be-6d9f835f9472 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:04.505539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734992d6-bb65-4132-9785-0ae6add57fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:04.506642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734992d6-bb65-4132-9785-0ae6add57fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:04.507136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734992d6-bb65-4132-9785-0ae6add57fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:04.507677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734992d6-bb65-4132-9785-0ae6add57fd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:04.525607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734992d6-bb65-4132-9785-0ae6add57fd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:04.528227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734992d6-bb65-4132-9785-0ae6add57fd1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-734992d6-bb65-4132-9785-0ae6add57fd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:04.548697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:04.550095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:04.551928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:04.562019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 246/487] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Aug 19 15:10:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:04.569666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932284-0215-4f38-8e1f-ea16304cc98c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:04.711655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df45d-48eb-424a-9cad-2a7681cdb10e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:04.712049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df45d-48eb-424a-9cad-2a7681cdb10e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:04.712572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862df45d-48eb-424a-9cad-2a7681cdb10e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:04.721534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 247/488] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:04 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:10:04.730306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932284-0215-4f38-8e1f-ea16304cc98c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:04.731088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bc79b-a517-421b-bddd-c33685b8aba7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:04.731487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bc79b-a517-421b-bddd-c33685b8aba7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:04.731868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bc79b-a517-421b-bddd-c33685b8aba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:04.761884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:04.769094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb26eb-7a23-4bfa-aad2-9b3effee1c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:04.769664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb26eb-7a23-4bfa-aad2-9b3effee1c53 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:04.769945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb26eb-7a23-4bfa-aad2-9b3effee1c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:04.770222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb26eb-7a23-4bfa-aad2-9b3effee1c53 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:04.770956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb26eb-7a23-4bfa-aad2-9b3effee1c53 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:04.771312 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb26eb-7a23-4bfa-aad2-9b3effee1c53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:04.773094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb26eb-7a23-4bfa-aad2-9b3effee1c53 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:04.797464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d03bc79b-a517-421b-bddd-c33685b8aba7 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:04.798705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d03bc79b-a517-421b-bddd-c33685b8aba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:04.815235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 242/489] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Aug 19 15:10:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:04.820342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb26eb-7a23-4bfa-aad2-9b3effee1c53 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:04.861845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:04.935941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb26eb-7a23-4bfa-aad2-9b3effee1c53 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:04.975403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0b434-061c-40eb-8692-272b6dfdd2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:04.976425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0b434-061c-40eb-8692-272b6dfdd2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:04.976975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0b434-061c-40eb-8692-272b6dfdd2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:04.977391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0b434-061c-40eb-8692-272b6dfdd2d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.027525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 248/490] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:05.031066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e34510-7bfa-41f9-8b63-e8c31c6fc4ad None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.031066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e34510-7bfa-41f9-8b63-e8c31c6fc4ad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.033871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:05.033991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e34510-7bfa-41f9-8b63-e8c31c6fc4ad None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.033991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 249/491] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:05.036827 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5355b49-5339-4384-a988-48476c6288e0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.037191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5355b49-5339-4384-a988-48476c6288e0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.037543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5355b49-5339-4384-a988-48476c6288e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.062224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b5355b49-5339-4384-a988-48476c6288e0 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:05.063944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b5355b49-5339-4384-a988-48476c6288e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:05.069610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0b434-061c-40eb-8692-272b6dfdd2d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:05.071278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0b434-061c-40eb-8692-272b6dfdd2d7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:05.155496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 250/492] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:05.157609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e28ef-8a45-45f9-bfc7-b7e77daf1259 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.157974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e28ef-8a45-45f9-bfc7-b7e77daf1259 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.158342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e28ef-8a45-45f9-bfc7-b7e77daf1259 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.160450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 251/493] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:05.165971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf89281-e450-43cb-86ad-52c311a73e92 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.166337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf89281-e450-43cb-86ad-52c311a73e92 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.167419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf89281-e450-43cb-86ad-52c311a73e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.187519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 243/494] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:10:05.192159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-eaf89281-e450-43cb-86ad-52c311a73e92 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:05.192159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eaf89281-e450-43cb-86ad-52c311a73e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:05.193542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0b434-061c-40eb-8692-272b6dfdd2d7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:05.277568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:05.293806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 252/495] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:10:05.316630 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0b434-061c-40eb-8692-272b6dfdd2d7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:05.355469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b483eec-c79f-4f37-af67-9ca65b258fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:05.356373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b483eec-c79f-4f37-af67-9ca65b258fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.356818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b483eec-c79f-4f37-af67-9ca65b258fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.357231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b483eec-c79f-4f37-af67-9ca65b258fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.368591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b483eec-c79f-4f37-af67-9ca65b258fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:05.371002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b483eec-c79f-4f37-af67-9ca65b258fc6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-1b483eec-c79f-4f37-af67-9ca65b258fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:05.391407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:05.392585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:05.393672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:05.393672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:05.393672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:05.395346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 244/496] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:05.401397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf89281-e450-43cb-86ad-52c311a73e92 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:05.434402 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:05.441046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b483eec-c79f-4f37-af67-9ca65b258fc6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:05.562760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf89281-e450-43cb-86ad-52c311a73e92 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:05.578198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b483eec-c79f-4f37-af67-9ca65b258fc6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:05.602462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:05.603097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f1a9e-4352-4ecf-9a83-707e3b1e64bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:05.603787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f1a9e-4352-4ecf-9a83-707e3b1e64bc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.604617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f1a9e-4352-4ecf-9a83-707e3b1e64bc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.604991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f1a9e-4352-4ecf-9a83-707e3b1e64bc None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.605819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f1a9e-4352-4ecf-9a83-707e3b1e64bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:05.606116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f1a9e-4352-4ecf-9a83-707e3b1e64bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:05.607519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f1a9e-4352-4ecf-9a83-707e3b1e64bc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:05.616321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83d4b-5575-4e5f-9a22-e7ef40aa7a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:05.616975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83d4b-5575-4e5f-9a22-e7ef40aa7a04 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.617260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83d4b-5575-4e5f-9a22-e7ef40aa7a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.617535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83d4b-5575-4e5f-9a22-e7ef40aa7a04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.624495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 253/497] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:05.624495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6ffb9-f29f-4c07-8171-8061f095d49e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.624495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6ffb9-f29f-4c07-8171-8061f095d49e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.624495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6ffb9-f29f-4c07-8171-8061f095d49e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.624495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 254/498] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:05.624495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6ffb9-f29f-4c07-8171-8061f095d49e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:05.635594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c83d4b-5575-4e5f-9a22-e7ef40aa7a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-99c83d4b-5575-4e5f-9a22-e7ef40aa7a04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:05.640936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:05.641859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:05.642691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 245/499] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:05.643371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83d4b-5575-4e5f-9a22-e7ef40aa7a04 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:05.714491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:05.737042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6ffb9-f29f-4c07-8171-8061f095d49e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:05.750911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83d4b-5575-4e5f-9a22-e7ef40aa7a04 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:05.771548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc698616-12e4-420f-80c2-e1059129aa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:05.772323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698616-12e4-420f-80c2-e1059129aa01 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.772712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698616-12e4-420f-80c2-e1059129aa01 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.772912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698616-12e4-420f-80c2-e1059129aa01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.783566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc698616-12e4-420f-80c2-e1059129aa01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:05.784489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d21c-136e-492e-8c78-57ffd54e3df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:05.785108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d21c-136e-492e-8c78-57ffd54e3df9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.785383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d21c-136e-492e-8c78-57ffd54e3df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.785650 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d21c-136e-492e-8c78-57ffd54e3df9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-fc698616-12e4-420f-80c2-e1059129aa01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:05.788215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:05.789272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:05.790058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 255/500] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:05.790739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ed3f-6a66-4ecf-87f1-caef412df695 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.790739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ed3f-6a66-4ecf-87f1-caef412df695 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.792441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ed3f-6a66-4ecf-87f1-caef412df695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.793756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1d21c-136e-492e-8c78-57ffd54e3df9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:05.795232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1d21c-136e-492e-8c78-57ffd54e3df9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5af1d21c-136e-492e-8c78-57ffd54e3df9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:05.808303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:05.809279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:05.810144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:05.810144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:05.810144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:05.810144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:05.810144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:05.810144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:05.810144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:05.810144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:05.810144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 246/501] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:05.810960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d21c-136e-492e-8c78-57ffd54e3df9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:05.814373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8835ed3f-6a66-4ecf-87f1-caef412df695 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:05.815497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8835ed3f-6a66-4ecf-87f1-caef412df695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:05.855983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:05.885181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 256/502] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:05.886066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835ed3f-6a66-4ecf-87f1-caef412df695 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:05.919191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d21c-136e-492e-8c78-57ffd54e3df9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:05.959421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17182402-5291-4c0a-bb80-8fb4912bacd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:05.960860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17182402-5291-4c0a-bb80-8fb4912bacd5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:05.961427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17182402-5291-4c0a-bb80-8fb4912bacd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:05.961883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17182402-5291-4c0a-bb80-8fb4912bacd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:05.968803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:05.974453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17182402-5291-4c0a-bb80-8fb4912bacd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-17182402-5291-4c0a-bb80-8fb4912bacd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:05.980354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:05.981287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:05.982099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 247/503] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:05.982755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17182402-5291-4c0a-bb80-8fb4912bacd5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.012150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835ed3f-6a66-4ecf-87f1-caef412df695 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.079244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643ed06-8baf-4779-9875-7e6c15944064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.080025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643ed06-8baf-4779-9875-7e6c15944064 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.080557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643ed06-8baf-4779-9875-7e6c15944064 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.081043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643ed06-8baf-4779-9875-7e6c15944064 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.082551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643ed06-8baf-4779-9875-7e6c15944064 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:06.082836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643ed06-8baf-4779-9875-7e6c15944064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.084974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643ed06-8baf-4779-9875-7e6c15944064 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.105273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 257/504] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:06.105273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643ed06-8baf-4779-9875-7e6c15944064 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.129090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17182402-5291-4c0a-bb80-8fb4912bacd5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.130856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:06.166267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1da2a-dd52-4587-a39c-f3c2c615d009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.167804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1da2a-dd52-4587-a39c-f3c2c615d009 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.168188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1da2a-dd52-4587-a39c-f3c2c615d009 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.168570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1da2a-dd52-4587-a39c-f3c2c615d009 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.169317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1da2a-dd52-4587-a39c-f3c2c615d009 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:06.169691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1da2a-dd52-4587-a39c-f3c2c615d009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.171177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1da2a-dd52-4587-a39c-f3c2c615d009 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.179792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 248/505] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:06.180689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1da2a-dd52-4587-a39c-f3c2c615d009 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.239192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643ed06-8baf-4779-9875-7e6c15944064 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.269942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:06.279541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf3d88-fa46-4b31-b438-906a9b44ed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.280158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf3d88-fa46-4b31-b438-906a9b44ed90 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.280470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf3d88-fa46-4b31-b438-906a9b44ed90 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.280752 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf3d88-fa46-4b31-b438-906a9b44ed90 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.281479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf3d88-fa46-4b31-b438-906a9b44ed90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:06.281786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf3d88-fa46-4b31-b438-906a9b44ed90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.283238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf3d88-fa46-4b31-b438-906a9b44ed90 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.287577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1da2a-dd52-4587-a39c-f3c2c615d009 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.302028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 258/506] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:06.302799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf3d88-fa46-4b31-b438-906a9b44ed90 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.329139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164234a1-9bda-4ed6-bdee-712487d4d178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.329816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164234a1-9bda-4ed6-bdee-712487d4d178 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.330096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164234a1-9bda-4ed6-bdee-712487d4d178 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.330368 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164234a1-9bda-4ed6-bdee-712487d4d178 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.331042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164234a1-9bda-4ed6-bdee-712487d4d178 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:06.331343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164234a1-9bda-4ed6-bdee-712487d4d178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.332814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164234a1-9bda-4ed6-bdee-712487d4d178 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.342246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 249/507] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:06.343094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164234a1-9bda-4ed6-bdee-712487d4d178 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.373342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:06.409395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf3d88-fa46-4b31-b438-906a9b44ed90 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.451544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666effd-6788-40a2-afb7-ff055a222aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.452181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666effd-6788-40a2-afb7-ff055a222aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.458306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666effd-6788-40a2-afb7-ff055a222aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.458306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4666effd-6788-40a2-afb7-ff055a222aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.458545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164234a1-9bda-4ed6-bdee-712487d4d178 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.464338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666effd-6788-40a2-afb7-ff055a222aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.466477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4666effd-6788-40a2-afb7-ff055a222aa6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4666effd-6788-40a2-afb7-ff055a222aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:06.482019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:06.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:06.484100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:06.484100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:06.484100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:06.484100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 259/508] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:06.484440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666effd-6788-40a2-afb7-ff055a222aa6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.500413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777c1a9-baad-455e-87d4-8a6b9979d658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.501062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777c1a9-baad-455e-87d4-8a6b9979d658 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.501355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777c1a9-baad-455e-87d4-8a6b9979d658 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.501626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777c1a9-baad-455e-87d4-8a6b9979d658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.513141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777c1a9-baad-455e-87d4-8a6b9979d658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.514643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777c1a9-baad-455e-87d4-8a6b9979d658 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8777c1a9-baad-455e-87d4-8a6b9979d658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:06.529449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:06.530512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:06.531529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:06.531529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:06.531529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:06.531529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 250/509] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:06.531795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777c1a9-baad-455e-87d4-8a6b9979d658 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.534774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:06.597659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4666effd-6788-40a2-afb7-ff055a222aa6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.611322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:06.631135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346a81c-53a1-439e-930c-f7cf681b17c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.631761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346a81c-53a1-439e-930c-f7cf681b17c1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.632038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346a81c-53a1-439e-930c-f7cf681b17c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.632311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346a81c-53a1-439e-930c-f7cf681b17c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.633408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777c1a9-baad-455e-87d4-8a6b9979d658 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.644820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346a81c-53a1-439e-930c-f7cf681b17c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.646299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346a81c-53a1-439e-930c-f7cf681b17c1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1346a81c-53a1-439e-930c-f7cf681b17c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:06.660675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:06.661631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:06.662489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:06.662489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:06.662489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:06.662489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 260/510] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:06.662489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346a81c-53a1-439e-930c-f7cf681b17c1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.672581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a7f91-468d-42d1-9c26-62c6bc238225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.673182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a7f91-468d-42d1-9c26-62c6bc238225 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.673467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a7f91-468d-42d1-9c26-62c6bc238225 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.673741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a7f91-468d-42d1-9c26-62c6bc238225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.674486 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a7f91-468d-42d1-9c26-62c6bc238225 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=98a63d5b605a4ecfad2e305c491ae755) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:06.674794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a7f91-468d-42d1-9c26-62c6bc238225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.676249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a7f91-468d-42d1-9c26-62c6bc238225 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.682029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 251/511] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/role_assignments?user.id=98a63d5b605a4ecfad2e305c491ae755&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:06.683360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a7f91-468d-42d1-9c26-62c6bc238225 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.774616 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:06.783338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346a81c-53a1-439e-930c-f7cf681b17c1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.810913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a7f91-468d-42d1-9c26-62c6bc238225 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.824381 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520864a-9746-405e-babb-2645179047cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.825048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520864a-9746-405e-babb-2645179047cf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.825331 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520864a-9746-405e-babb-2645179047cf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.825603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520864a-9746-405e-babb-2645179047cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.836901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520864a-9746-405e-babb-2645179047cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.838411 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520864a-9746-405e-babb-2645179047cf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3520864a-9746-405e-babb-2645179047cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:06.852643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:06.853560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:06.854451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:06.854451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:06.854451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:06.854451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 261/512] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:06.855161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520864a-9746-405e-babb-2645179047cf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.859605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2439f-a9dc-459b-b1cd-8448349a7fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.860265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2439f-a9dc-459b-b1cd-8448349a7fef None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.860553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2439f-a9dc-459b-b1cd-8448349a7fef None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:06.860829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2439f-a9dc-459b-b1cd-8448349a7fef None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:06.861481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2439f-a9dc-459b-b1cd-8448349a7fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:06.861788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2439f-a9dc-459b-b1cd-8448349a7fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:06.863174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2439f-a9dc-459b-b1cd-8448349a7fef None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:06.880888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 252/513] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:06.880888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2439f-a9dc-459b-b1cd-8448349a7fef None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:06.883996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:06.962393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520864a-9746-405e-babb-2645179047cf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.996018 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2439f-a9dc-459b-b1cd-8448349a7fef None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:06.998954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bd883-96b9-45fe-8206-5cfb3bd4eea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:06.999567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bd883-96b9-45fe-8206-5cfb3bd4eea3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:06.999848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bd883-96b9-45fe-8206-5cfb3bd4eea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.000123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bd883-96b9-45fe-8206-5cfb3bd4eea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.000817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bd883-96b9-45fe-8206-5cfb3bd4eea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:07.001115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bd883-96b9-45fe-8206-5cfb3bd4eea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.002534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bd883-96b9-45fe-8206-5cfb3bd4eea3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.020728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 262/514] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:07.020728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bd883-96b9-45fe-8206-5cfb3bd4eea3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.029472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:07.038052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b4467-6691-488f-8f46-30e96fa0c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.038660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b4467-6691-488f-8f46-30e96fa0c425 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.038944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b4467-6691-488f-8f46-30e96fa0c425 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.039274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b4467-6691-488f-8f46-30e96fa0c425 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.039867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b4467-6691-488f-8f46-30e96fa0c425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:07.040172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b4467-6691-488f-8f46-30e96fa0c425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.042155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b4467-6691-488f-8f46-30e96fa0c425 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.058452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 253/515] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:07.058452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b4467-6691-488f-8f46-30e96fa0c425 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.119401 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:07.132088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bd883-96b9-45fe-8206-5cfb3bd4eea3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.164358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b4467-6691-488f-8f46-30e96fa0c425 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad5393-6c87-49a3-bc69-56f8f1dbafe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.168174 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad5393-6c87-49a3-bc69-56f8f1dbafe2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.168477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad5393-6c87-49a3-bc69-56f8f1dbafe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.168754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad5393-6c87-49a3-bc69-56f8f1dbafe2 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.169394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad5393-6c87-49a3-bc69-56f8f1dbafe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:07.169693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad5393-6c87-49a3-bc69-56f8f1dbafe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.171080 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad5393-6c87-49a3-bc69-56f8f1dbafe2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.183235 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 263/516] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:07.184143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad5393-6c87-49a3-bc69-56f8f1dbafe2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.205704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10976769-7775-40bf-ad4d-edeb61890460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.206338 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10976769-7775-40bf-ad4d-edeb61890460 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.206629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10976769-7775-40bf-ad4d-edeb61890460 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.206910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10976769-7775-40bf-ad4d-edeb61890460 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.215412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10976769-7775-40bf-ad4d-edeb61890460 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.216924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10976769-7775-40bf-ad4d-edeb61890460 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-10976769-7775-40bf-ad4d-edeb61890460 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:07.229943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:07.230970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 254/517] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:07.231864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10976769-7775-40bf-ad4d-edeb61890460 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.258243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:07.296074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad5393-6c87-49a3-bc69-56f8f1dbafe2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.336130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf275e6-7a2d-4aaa-9604-93fe33d06beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.336822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf275e6-7a2d-4aaa-9604-93fe33d06beb None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.337204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf275e6-7a2d-4aaa-9604-93fe33d06beb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.337372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf275e6-7a2d-4aaa-9604-93fe33d06beb None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.338015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf275e6-7a2d-4aaa-9604-93fe33d06beb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.339598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf275e6-7a2d-4aaa-9604-93fe33d06beb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.343707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10976769-7775-40bf-ad4d-edeb61890460 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.364112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 264/518] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:10:07 2022] POST /identity/v3/users => generated 247 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:07.371193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:07.373582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf275e6-7a2d-4aaa-9604-93fe33d06beb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.387322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33b41c-54fd-4c43-a87a-fd2fbccd3260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.387842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33b41c-54fd-4c43-a87a-fd2fbccd3260 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.388136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33b41c-54fd-4c43-a87a-fd2fbccd3260 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.388423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33b41c-54fd-4c43-a87a-fd2fbccd3260 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.398312 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33b41c-54fd-4c43-a87a-fd2fbccd3260 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.399886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33b41c-54fd-4c43-a87a-fd2fbccd3260 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5a33b41c-54fd-4c43-a87a-fd2fbccd3260 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:07.414414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:07.415426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 255/519] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:07.417618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33b41c-54fd-4c43-a87a-fd2fbccd3260 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.499936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf275e6-7a2d-4aaa-9604-93fe33d06beb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.532028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:07.541625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33b41c-54fd-4c43-a87a-fd2fbccd3260 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.546343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b593a-d70e-43ce-aa45-008ad4ac8cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.547247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b593a-d70e-43ce-aa45-008ad4ac8cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.547727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b593a-d70e-43ce-aa45-008ad4ac8cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.548197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b593a-d70e-43ce-aa45-008ad4ac8cd0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.556944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b593a-d70e-43ce-aa45-008ad4ac8cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.558567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b593a-d70e-43ce-aa45-008ad4ac8cd0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e28b593a-d70e-43ce-aa45-008ad4ac8cd0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:07.571043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:07.572156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 15:10:07.573396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:07.576485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 265/520] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:07.576485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b593a-d70e-43ce-aa45-008ad4ac8cd0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.595836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adadf3-96f9-4182-84be-d4920e212691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.596696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adadf3-96f9-4182-84be-d4920e212691 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.596976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adadf3-96f9-4182-84be-d4920e212691 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.597296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adadf3-96f9-4182-84be-d4920e212691 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.598336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adadf3-96f9-4182-84be-d4920e212691 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:07.599041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adadf3-96f9-4182-84be-d4920e212691 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.600531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adadf3-96f9-4182-84be-d4920e212691 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.627018 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 256/521] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Aug 19 15:10:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:07.627946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adadf3-96f9-4182-84be-d4920e212691 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.690750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:07.700809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b593a-d70e-43ce-aa45-008ad4ac8cd0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.742120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ee391-45a2-4dc1-8a6f-fd8d3d548d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.743076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ee391-45a2-4dc1-8a6f-fd8d3d548d3e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.743598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ee391-45a2-4dc1-8a6f-fd8d3d548d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.744144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ee391-45a2-4dc1-8a6f-fd8d3d548d3e None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.745412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ee391-45a2-4dc1-8a6f-fd8d3d548d3e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:07.745924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ee391-45a2-4dc1-8a6f-fd8d3d548d3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.747654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ee391-45a2-4dc1-8a6f-fd8d3d548d3e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.766007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adadf3-96f9-4182-84be-d4920e212691 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.791758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 266/522] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Aug 19 15:10:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:07.793102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ee391-45a2-4dc1-8a6f-fd8d3d548d3e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.811880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b7971-4ef2-4c40-88be-c8374f94574c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.813155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b7971-4ef2-4c40-88be-c8374f94574c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.813663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b7971-4ef2-4c40-88be-c8374f94574c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.814135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b7971-4ef2-4c40-88be-c8374f94574c None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:07.815132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b7971-4ef2-4c40-88be-c8374f94574c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:07.815690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b7971-4ef2-4c40-88be-c8374f94574c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:07.817383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b7971-4ef2-4c40-88be-c8374f94574c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:07.840616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 257/523] 10.222.0.75 () {56 vars in 1139 bytes} [Fri Aug 19 15:10:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:07.841867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b7971-4ef2-4c40-88be-c8374f94574c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:07.851050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:07.933593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ee391-45a2-4dc1-8a6f-fd8d3d548d3e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.959346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:07.965608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b7971-4ef2-4c40-88be-c8374f94574c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:07.975083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24285db3-181a-44d7-8b49-88e23596d6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:07.975694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24285db3-181a-44d7-8b49-88e23596d6ce None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:07.976003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24285db3-181a-44d7-8b49-88e23596d6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:07.976262 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24285db3-181a-44d7-8b49-88e23596d6ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.006500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566beff8-96ae-46a6-87e5-b2ab1a451334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:08.007502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566beff8-96ae-46a6-87e5-b2ab1a451334 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.007985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566beff8-96ae-46a6-87e5-b2ab1a451334 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.008582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566beff8-96ae-46a6-87e5-b2ab1a451334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.048820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24285db3-181a-44d7-8b49-88e23596d6ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:08.050307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24285db3-181a-44d7-8b49-88e23596d6ce None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:08.087345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566beff8-96ae-46a6-87e5-b2ab1a451334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:08.090072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566beff8-96ae-46a6-87e5-b2ab1a451334 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:08.094968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:08.163920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 267/524] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:10:08.164916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24285db3-181a-44d7-8b49-88e23596d6ce None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:08.198808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:08.204994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 258/525] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:10:08.206178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566beff8-96ae-46a6-87e5-b2ab1a451334 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:08.275800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24285db3-181a-44d7-8b49-88e23596d6ce None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:08.326084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e935c-3c37-4cdd-9f21-479d57e9cdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:08.326887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e935c-3c37-4cdd-9f21-479d57e9cdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.327754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e935c-3c37-4cdd-9f21-479d57e9cdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.327754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e935c-3c37-4cdd-9f21-479d57e9cdb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.332782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566beff8-96ae-46a6-87e5-b2ab1a451334 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:08.350104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:08.391362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ec97b-dbd0-49db-b6ba-034add9a3352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:08.391994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ec97b-dbd0-49db-b6ba-034add9a3352 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.392274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ec97b-dbd0-49db-b6ba-034add9a3352 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.392688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ec97b-dbd0-49db-b6ba-034add9a3352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.403579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ec97b-dbd0-49db-b6ba-034add9a3352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:08.405170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ec97b-dbd0-49db-b6ba-034add9a3352 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:08.420187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e935c-3c37-4cdd-9f21-479d57e9cdb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-896ec97b-dbd0-49db-b6ba-034add9a3352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:08.422079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:08.423704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:08.424947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:08.424947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:08.424947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:08.426197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e935c-3c37-4cdd-9f21-479d57e9cdb3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:08.428507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 259/526] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:08.429362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ec97b-dbd0-49db-b6ba-034add9a3352 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:08.482556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:08.537113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ec97b-dbd0-49db-b6ba-034add9a3352 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:08.554919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 268/527] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:10:08.556315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e935c-3c37-4cdd-9f21-479d57e9cdb3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:08.574179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf1c1b-223d-469c-80c1-a0a0b8e725f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:08.574843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf1c1b-223d-469c-80c1-a0a0b8e725f9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.575083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf1c1b-223d-469c-80c1-a0a0b8e725f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.575359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf1c1b-223d-469c-80c1-a0a0b8e725f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.586257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf1c1b-223d-469c-80c1-a0a0b8e725f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:08.587742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf1c1b-223d-469c-80c1-a0a0b8e725f9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-41cf1c1b-223d-469c-80c1-a0a0b8e725f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:08.602528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:08.603498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:08.604938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:08.604938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:08.604938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:08.604938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 260/528] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:08.604938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf1c1b-223d-469c-80c1-a0a0b8e725f9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:08.609438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:08.679214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e935c-3c37-4cdd-9f21-479d57e9cdb3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:08.715085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188bcb8-7713-4e32-955b-545f7010c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:08.715698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188bcb8-7713-4e32-955b-545f7010c08a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.715947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188bcb8-7713-4e32-955b-545f7010c08a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.716216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188bcb8-7713-4e32-955b-545f7010c08a None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.716886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188bcb8-7713-4e32-955b-545f7010c08a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:08.717184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188bcb8-7713-4e32-955b-545f7010c08a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:08.718349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf1c1b-223d-469c-80c1-a0a0b8e725f9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:08.718610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188bcb8-7713-4e32-955b-545f7010c08a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:08.720143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:08.742734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 269/529] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:08.742734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188bcb8-7713-4e32-955b-545f7010c08a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:08.766240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e16a15-3f4a-48cc-b552-71cb9009e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:08.766964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e16a15-3f4a-48cc-b552-71cb9009e462 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.767245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e16a15-3f4a-48cc-b552-71cb9009e462 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.767518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e16a15-3f4a-48cc-b552-71cb9009e462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.778912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e16a15-3f4a-48cc-b552-71cb9009e462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:08.780405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e16a15-3f4a-48cc-b552-71cb9009e462 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f6e16a15-3f4a-48cc-b552-71cb9009e462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:08.795943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:08.796920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:08.797757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:08.797757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:08.797757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:08.797757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 261/530] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:08.798394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ef213-dddf-4a0a-86b1-690803cf1fab None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.798667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ef213-dddf-4a0a-86b1-690803cf1fab None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.799074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ef213-dddf-4a0a-86b1-690803cf1fab None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.804182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 262/531] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:08.804965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ef213-dddf-4a0a-86b1-690803cf1fab None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:08.850157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188bcb8-7713-4e32-955b-545f7010c08a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:08.852188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:08.889433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece673f-05d7-4c47-a1ec-57412f998ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:08.890398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece673f-05d7-4c47-a1ec-57412f998ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.890683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece673f-05d7-4c47-a1ec-57412f998ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.890958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece673f-05d7-4c47-a1ec-57412f998ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.891913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece673f-05d7-4c47-a1ec-57412f998ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:08.892402 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece673f-05d7-4c47-a1ec-57412f998ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:08.893806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece673f-05d7-4c47-a1ec-57412f998ae7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:08.907197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 270/532] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:08.908009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece673f-05d7-4c47-a1ec-57412f998ae7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:08.921259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ef213-dddf-4a0a-86b1-690803cf1fab None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:08.968416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35021371-651b-4c38-98fe-bcc2f29d041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:08.969041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021371-651b-4c38-98fe-bcc2f29d041d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.969582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021371-651b-4c38-98fe-bcc2f29d041d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.969850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35021371-651b-4c38-98fe-bcc2f29d041d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:08.977498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35021371-651b-4c38-98fe-bcc2f29d041d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:08.979013 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35021371-651b-4c38-98fe-bcc2f29d041d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-35021371-651b-4c38-98fe-bcc2f29d041d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:08.991750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:08.992674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 263/533] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e04658-3bb0-425f-b678-1cc657374836 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:08.999153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e04658-3bb0-425f-b678-1cc657374836 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:08.999747 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e04658-3bb0-425f-b678-1cc657374836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.012952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f0e04658-3bb0-425f-b678-1cc657374836 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:09.014136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f0e04658-3bb0-425f-b678-1cc657374836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:09.023932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:09.030312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece673f-05d7-4c47-a1ec-57412f998ae7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:09.067312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586407e1-8109-4b9f-b0fa-b4af810ca518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:09.067922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586407e1-8109-4b9f-b0fa-b4af810ca518 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.068193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586407e1-8109-4b9f-b0fa-b4af810ca518 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.068488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586407e1-8109-4b9f-b0fa-b4af810ca518 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.069130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586407e1-8109-4b9f-b0fa-b4af810ca518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:09.069429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586407e1-8109-4b9f-b0fa-b4af810ca518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:09.070808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586407e1-8109-4b9f-b0fa-b4af810ca518 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:09.087644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 271/534] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:09.087644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586407e1-8109-4b9f-b0fa-b4af810ca518 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.090117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 264/535] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:09.090793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e04658-3bb0-425f-b678-1cc657374836 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.127201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:09.203898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e04658-3bb0-425f-b678-1cc657374836 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:09.235980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586407e1-8109-4b9f-b0fa-b4af810ca518 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:09.248896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e11876-393f-4d78-81a8-74d94552ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:09.250436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e11876-393f-4d78-81a8-74d94552ea00 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.250899 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e11876-393f-4d78-81a8-74d94552ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.251363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e11876-393f-4d78-81a8-74d94552ea00 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.252516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e11876-393f-4d78-81a8-74d94552ea00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:09.253470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e11876-393f-4d78-81a8-74d94552ea00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:09.255698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e11876-393f-4d78-81a8-74d94552ea00 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:09.274522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410763ac-e631-4331-a965-1844e833eb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:09.275275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410763ac-e631-4331-a965-1844e833eb7b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.275621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410763ac-e631-4331-a965-1844e833eb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.276583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410763ac-e631-4331-a965-1844e833eb7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.282184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 265/536] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:09.283211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e11876-393f-4d78-81a8-74d94552ea00 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.286989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:09.289472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410763ac-e631-4331-a965-1844e833eb7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:09.291819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410763ac-e631-4331-a965-1844e833eb7b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-410763ac-e631-4331-a965-1844e833eb7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:09.305406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:09.306285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:09.307170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:09.307170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:09.307170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:09.307170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:09.307170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:09.307170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:09.307170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:09.307170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:09.311383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 272/537] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:09.312232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410763ac-e631-4331-a965-1844e833eb7b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.399731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e11876-393f-4d78-81a8-74d94552ea00 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:09.411179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:09.429120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410763ac-e631-4331-a965-1844e833eb7b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:09.434450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7718d4-d395-486d-b0f1-0be3e1999af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:09.435071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7718d4-d395-486d-b0f1-0be3e1999af0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.435351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7718d4-d395-486d-b0f1-0be3e1999af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.435624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7718d4-d395-486d-b0f1-0be3e1999af0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.442432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7718d4-d395-486d-b0f1-0be3e1999af0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:09.443889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7718d4-d395-486d-b0f1-0be3e1999af0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3a7718d4-d395-486d-b0f1-0be3e1999af0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:09.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:09.456243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:09.457125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:09.457125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:09.457125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:09.457125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:09.457125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:09.457125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:09.457125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:09.457125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:09.457125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 266/538] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:09.457673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7718d4-d395-486d-b0f1-0be3e1999af0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.466978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f5d22-9cfb-43f8-b6c3-4c79b2a90388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:09.467588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f5d22-9cfb-43f8-b6c3-4c79b2a90388 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.467870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f5d22-9cfb-43f8-b6c3-4c79b2a90388 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.468150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f5d22-9cfb-43f8-b6c3-4c79b2a90388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.468909 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f5d22-9cfb-43f8-b6c3-4c79b2a90388 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=de0dfcbad22348fdaea32d64557e68a6) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:09.469219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f5d22-9cfb-43f8-b6c3-4c79b2a90388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:09.470668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f5d22-9cfb-43f8-b6c3-4c79b2a90388 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:09.476153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 273/539] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/role_assignments?user.id=de0dfcbad22348fdaea32d64557e68a6&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:09.476972 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f5d22-9cfb-43f8-b6c3-4c79b2a90388 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.550458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:09.596407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f5d22-9cfb-43f8-b6c3-4c79b2a90388 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:09.597161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7718d4-d395-486d-b0f1-0be3e1999af0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:09.634256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55b0a1-8a24-457d-a242-fe97827de23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:09.634895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55b0a1-8a24-457d-a242-fe97827de23b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.635176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55b0a1-8a24-457d-a242-fe97827de23b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.636183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55b0a1-8a24-457d-a242-fe97827de23b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.640965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7097a5-0fc3-4942-b974-b4d11d027678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:09.642020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7097a5-0fc3-4942-b974-b4d11d027678 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.642598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7097a5-0fc3-4942-b974-b4d11d027678 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.643041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7097a5-0fc3-4942-b974-b4d11d027678 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.644519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7097a5-0fc3-4942-b974-b4d11d027678 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:09.644996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7097a5-0fc3-4942-b974-b4d11d027678 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:09.646881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7097a5-0fc3-4942-b974-b4d11d027678 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:09.646999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55b0a1-8a24-457d-a242-fe97827de23b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bc55b0a1-8a24-457d-a242-fe97827de23b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:09.652451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:09.653549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 274/540] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:09.659244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18bd17-7004-4c74-bb68-dac2b087a4a3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.660144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18bd17-7004-4c74-bb68-dac2b087a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.660144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18bd17-7004-4c74-bb68-dac2b087a4a3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.660144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 275/541] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:09.660144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18bd17-7004-4c74-bb68-dac2b087a4a3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.677328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 267/542] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:09.677328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7097a5-0fc3-4942-b974-b4d11d027678 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.677954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:09.783275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18bd17-7004-4c74-bb68-dac2b087a4a3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:09.799691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7097a5-0fc3-4942-b974-b4d11d027678 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:09.817720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bb5cb-50d9-4705-9c66-6c149094d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:09.818251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bb5cb-50d9-4705-9c66-6c149094d16e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.818530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bb5cb-50d9-4705-9c66-6c149094d16e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.818804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bb5cb-50d9-4705-9c66-6c149094d16e None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.819491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bb5cb-50d9-4705-9c66-6c149094d16e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:09.819794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bb5cb-50d9-4705-9c66-6c149094d16e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:09.821313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bb5cb-50d9-4705-9c66-6c149094d16e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:09.827222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:09.834268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 276/543] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:09.835467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20486f7a-8f52-4a86-8e5d-efff5f0b4e98 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.835752 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20486f7a-8f52-4a86-8e5d-efff5f0b4e98 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.836022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20486f7a-8f52-4a86-8e5d-efff5f0b4e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.844510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62ebb4-1543-4ec0-9fc8-60f9d581485d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:09.845466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62ebb4-1543-4ec0-9fc8-60f9d581485d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:09.845951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62ebb4-1543-4ec0-9fc8-60f9d581485d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:09.846435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62ebb4-1543-4ec0-9fc8-60f9d581485d None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:09.847300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62ebb4-1543-4ec0-9fc8-60f9d581485d None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:09.847799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62ebb4-1543-4ec0-9fc8-60f9d581485d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:09.849441 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62ebb4-1543-4ec0-9fc8-60f9d581485d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:09.853793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-20486f7a-8f52-4a86-8e5d-efff5f0b4e98 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:09.855288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-20486f7a-8f52-4a86-8e5d-efff5f0b4e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:09.858285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 268/544] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:09.859287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62ebb4-1543-4ec0-9fc8-60f9d581485d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.928618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:09.937311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 277/545] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:09.943418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20486f7a-8f52-4a86-8e5d-efff5f0b4e98 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:09.977445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62ebb4-1543-4ec0-9fc8-60f9d581485d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:10.021821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a68c3-c3e5-4fb7-a0ec-7dfc2d418833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:10.022692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a68c3-c3e5-4fb7-a0ec-7dfc2d418833 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:10.022983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a68c3-c3e5-4fb7-a0ec-7dfc2d418833 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:10.023262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a68c3-c3e5-4fb7-a0ec-7dfc2d418833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:10.024003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a68c3-c3e5-4fb7-a0ec-7dfc2d418833 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=a6b4d6a93c7e49009aca527e281b5c3e) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:10.024314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a68c3-c3e5-4fb7-a0ec-7dfc2d418833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:10.025783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a68c3-c3e5-4fb7-a0ec-7dfc2d418833 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:10.034096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 269/546] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/role_assignments?user.id=a6b4d6a93c7e49009aca527e281b5c3e&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:10.037349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a68c3-c3e5-4fb7-a0ec-7dfc2d418833 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:10.102583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20486f7a-8f52-4a86-8e5d-efff5f0b4e98 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:10.114933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:10.180872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da8e4d-8de6-47d5-b9f1-3f56c23949ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:10.181923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da8e4d-8de6-47d5-b9f1-3f56c23949ec None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:10.183207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da8e4d-8de6-47d5-b9f1-3f56c23949ec None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:10.185056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da8e4d-8de6-47d5-b9f1-3f56c23949ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:10.186109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da8e4d-8de6-47d5-b9f1-3f56c23949ec None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=c8704aae85084d41bdedb5537ef96e44) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:10.186938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da8e4d-8de6-47d5-b9f1-3f56c23949ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:10.189446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da8e4d-8de6-47d5-b9f1-3f56c23949ec None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:10.204719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 278/547] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:09 2022] GET /identity/v3/role_assignments?user.id=c8704aae85084d41bdedb5537ef96e44&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:10.219428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da8e4d-8de6-47d5-b9f1-3f56c23949ec None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:10.241710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a68c3-c3e5-4fb7-a0ec-7dfc2d418833 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:10.290103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507413df-456f-4861-aab3-f8f9c5f120bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:10.290103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507413df-456f-4861-aab3-f8f9c5f120bf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:10.290103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507413df-456f-4861-aab3-f8f9c5f120bf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:10.290103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507413df-456f-4861-aab3-f8f9c5f120bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:10.314700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507413df-456f-4861-aab3-f8f9c5f120bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:10.316526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507413df-456f-4861-aab3-f8f9c5f120bf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:10.324798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-507413df-456f-4861-aab3-f8f9c5f120bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:10.347394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:10.348561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:10.349703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:10.349703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:10.349703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:10.349703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 270/548] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:10.357139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507413df-456f-4861-aab3-f8f9c5f120bf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:10.396557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da8e4d-8de6-47d5-b9f1-3f56c23949ec None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:10.447132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6da373-37d4-40a3-8d01-b01409af7912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:10.448051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6da373-37d4-40a3-8d01-b01409af7912 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:10.448509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6da373-37d4-40a3-8d01-b01409af7912 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:10.448902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6da373-37d4-40a3-8d01-b01409af7912 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:10.468241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6da373-37d4-40a3-8d01-b01409af7912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bc6da373-37d4-40a3-8d01-b01409af7912 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:10.473949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:10.475425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:10.476915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:10.477974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 279/549] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:10.488861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6da373-37d4-40a3-8d01-b01409af7912 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:10.544955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507413df-456f-4861-aab3-f8f9c5f120bf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:10.548800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:10.600738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022f6c-c405-40a4-b12b-d63fbfc0ef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:10.601523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022f6c-c405-40a4-b12b-d63fbfc0ef28 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:10.601896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022f6c-c405-40a4-b12b-d63fbfc0ef28 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:10.602267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022f6c-c405-40a4-b12b-d63fbfc0ef28 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:10.603048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022f6c-c405-40a4-b12b-d63fbfc0ef28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:10.603448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022f6c-c405-40a4-b12b-d63fbfc0ef28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:10.605678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022f6c-c405-40a4-b12b-d63fbfc0ef28 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:10.628727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 271/550] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:10.635691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022f6c-c405-40a4-b12b-d63fbfc0ef28 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:10.681473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6da373-37d4-40a3-8d01-b01409af7912 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:10.722283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:10.747075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca2a91-b838-4cbe-8901-f545e6a66fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:10.747979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca2a91-b838-4cbe-8901-f545e6a66fdb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:10.748464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca2a91-b838-4cbe-8901-f545e6a66fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:10.748937 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca2a91-b838-4cbe-8901-f545e6a66fdb None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:10.749780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca2a91-b838-4cbe-8901-f545e6a66fdb None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:10.750276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca2a91-b838-4cbe-8901-f545e6a66fdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:10.751830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca2a91-b838-4cbe-8901-f545e6a66fdb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:10.765617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 280/551] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:10.767037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca2a91-b838-4cbe-8901-f545e6a66fdb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:10.855892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022f6c-c405-40a4-b12b-d63fbfc0ef28 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:10.910587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca2a91-b838-4cbe-8901-f545e6a66fdb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:10.941773 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004ac88-a28c-4e53-a642-837de76c31bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:10.942417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004ac88-a28c-4e53-a642-837de76c31bd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:10.942698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004ac88-a28c-4e53-a642-837de76c31bd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:10.942970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004ac88-a28c-4e53-a642-837de76c31bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:10.960496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004ac88-a28c-4e53-a642-837de76c31bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:10.961988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004ac88-a28c-4e53-a642-837de76c31bd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8004ac88-a28c-4e53-a642-837de76c31bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:10.981407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:10.983115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:10.984319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 272/552] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Aug 19 15:10:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:10.988545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004ac88-a28c-4e53-a642-837de76c31bd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:10.989019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d166be2-94c7-4772-8c4c-10bcfca10b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:10.989873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d166be2-94c7-4772-8c4c-10bcfca10b72 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:10.990298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d166be2-94c7-4772-8c4c-10bcfca10b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:10.990732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d166be2-94c7-4772-8c4c-10bcfca10b72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:11.008458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d166be2-94c7-4772-8c4c-10bcfca10b72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:11.010628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d166be2-94c7-4772-8c4c-10bcfca10b72 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:11.017169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3d166be2-94c7-4772-8c4c-10bcfca10b72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:11.028405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:11.029448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:11.030339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:11.030339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:11.030339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:11.032277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 281/553] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:11.041982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d166be2-94c7-4772-8c4c-10bcfca10b72 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:11.103171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004ac88-a28c-4e53-a642-837de76c31bd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:11.140992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731202-5216-4161-afb5-8f90c099a0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:11.141600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731202-5216-4161-afb5-8f90c099a0ff None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:11.141885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731202-5216-4161-afb5-8f90c099a0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:11.142158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731202-5216-4161-afb5-8f90c099a0ff None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:11.142898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731202-5216-4161-afb5-8f90c099a0ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:11.143207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731202-5216-4161-afb5-8f90c099a0ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:11.144675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731202-5216-4161-afb5-8f90c099a0ff None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:11.154957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:11.165170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 273/554] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Aug 19 15:10:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:11.172048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731202-5216-4161-afb5-8f90c099a0ff None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:11.271545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:11.282724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731202-5216-4161-afb5-8f90c099a0ff None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:11.332893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6076bc-123a-4a84-8e00-4182d496f6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:11.333528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6076bc-123a-4a84-8e00-4182d496f6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:11.333807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6076bc-123a-4a84-8e00-4182d496f6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:11.334080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6076bc-123a-4a84-8e00-4182d496f6b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:11.350768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6076bc-123a-4a84-8e00-4182d496f6b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:11.355913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6076bc-123a-4a84-8e00-4182d496f6b2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:11.380909 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d166be2-94c7-4772-8c4c-10bcfca10b72 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0e6076bc-123a-4a84-8e00-4182d496f6b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:11.394042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:11.395215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:11.396872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:11.396872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:11.396872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:11.396872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 274/555] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:11.404106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6076bc-123a-4a84-8e00-4182d496f6b2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:11.437121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643fa13-2c97-45b5-9f3b-d33211ec2f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:11.437656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643fa13-2c97-45b5-9f3b-d33211ec2f34 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:11.437935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643fa13-2c97-45b5-9f3b-d33211ec2f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:11.438209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643fa13-2c97-45b5-9f3b-d33211ec2f34 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:11.439331 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643fa13-2c97-45b5-9f3b-d33211ec2f34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:11.439638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643fa13-2c97-45b5-9f3b-d33211ec2f34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:11.441279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643fa13-2c97-45b5-9f3b-d33211ec2f34 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:11.461320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 282/556] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:11.464524 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643fa13-2c97-45b5-9f3b-d33211ec2f34 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:11.580767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6076bc-123a-4a84-8e00-4182d496f6b2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:11.603610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:11.612458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643fa13-2c97-45b5-9f3b-d33211ec2f34 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:11.623665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:11.633825 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a3fed-1794-447a-9405-cfabb6999be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:11.634439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a3fed-1794-447a-9405-cfabb6999be1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:11.634721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a3fed-1794-447a-9405-cfabb6999be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:11.634996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a3fed-1794-447a-9405-cfabb6999be1 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:11.636184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a3fed-1794-447a-9405-cfabb6999be1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:11.636518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a3fed-1794-447a-9405-cfabb6999be1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:11.638164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a3fed-1794-447a-9405-cfabb6999be1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:11.650707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 275/557] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:11.657016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8108e69-b3e8-4a68-9945-c2045be922ef None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:11.657321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8108e69-b3e8-4a68-9945-c2045be922ef None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:11.657604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8108e69-b3e8-4a68-9945-c2045be922ef None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:11.659109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46622e-b487-44fa-a661-b5cb2399f459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:11.659238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 276/558] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:11.663396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8108e69-b3e8-4a68-9945-c2045be922ef None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:11.664274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46622e-b487-44fa-a661-b5cb2399f459 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:11.664821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46622e-b487-44fa-a661-b5cb2399f459 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:11.665295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46622e-b487-44fa-a661-b5cb2399f459 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:11.685151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46622e-b487-44fa-a661-b5cb2399f459 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:11.687076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46622e-b487-44fa-a661-b5cb2399f459 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bd46622e-b487-44fa-a661-b5cb2399f459 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:11.700459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:11.701453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:11.702398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:11.705261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 283/559] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:11.706929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4daa7-5c5a-4445-a7ce-91d6b8f2e6ae None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:11.709106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4daa7-5c5a-4445-a7ce-91d6b8f2e6ae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:11.709643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4daa7-5c5a-4445-a7ce-91d6b8f2e6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:11.733979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-edb4daa7-5c5a-4445-a7ce-91d6b8f2e6ae None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:11.735571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-edb4daa7-5c5a-4445-a7ce-91d6b8f2e6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:11.783726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8108e69-b3e8-4a68-9945-c2045be922ef None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:11.816658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 284/560] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:11.818001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4daa7-5c5a-4445-a7ce-91d6b8f2e6ae None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:11.823959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6c42e-5918-475b-8738-9040079733e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:11.824359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6c42e-5918-475b-8738-9040079733e2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:11.824672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6c42e-5918-475b-8738-9040079733e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:11.824944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6c42e-5918-475b-8738-9040079733e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:11.832797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6c42e-5918-475b-8738-9040079733e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:11.834439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6c42e-5918-475b-8738-9040079733e2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e2c6c42e-5918-475b-8738-9040079733e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:11.848535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:11.849543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:11.850456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:11.850456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:11.850456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:11.850456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:11.850456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:11.850456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:11.850456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:11.850456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:11.850456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 277/561] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:11.855664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6c42e-5918-475b-8738-9040079733e2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:11.913156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:11.967951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6c42e-5918-475b-8738-9040079733e2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:11.975804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:12.005611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4daa7-5c5a-4445-a7ce-91d6b8f2e6ae None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:12.007004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6347f38-7f98-45f0-9f4f-67df27632eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:12.008144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6347f38-7f98-45f0-9f4f-67df27632eee None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.008701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6347f38-7f98-45f0-9f4f-67df27632eee None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.009181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6347f38-7f98-45f0-9f4f-67df27632eee None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.010089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6347f38-7f98-45f0-9f4f-67df27632eee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:12.011299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6347f38-7f98-45f0-9f4f-67df27632eee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:12.013142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6347f38-7f98-45f0-9f4f-67df27632eee None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:12.028210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 278/562] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:12.039186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6347f38-7f98-45f0-9f4f-67df27632eee None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:12.097149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d3fde-477a-4d03-88e9-f4d5e03ca23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:12.097899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d3fde-477a-4d03-88e9-f4d5e03ca23b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.098260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d3fde-477a-4d03-88e9-f4d5e03ca23b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.098604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d3fde-477a-4d03-88e9-f4d5e03ca23b None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.099479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d3fde-477a-4d03-88e9-f4d5e03ca23b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:12.099766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d3fde-477a-4d03-88e9-f4d5e03ca23b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:12.101584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d3fde-477a-4d03-88e9-f4d5e03ca23b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:12.137659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 285/563] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Aug 19 15:10:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:12.142926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d3fde-477a-4d03-88e9-f4d5e03ca23b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:12.159304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6347f38-7f98-45f0-9f4f-67df27632eee None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:12.173907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:12.198967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c025dc8-3065-4707-93c0-7ef6a3373096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:12.199921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c025dc8-3065-4707-93c0-7ef6a3373096 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.200438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c025dc8-3065-4707-93c0-7ef6a3373096 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.200895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c025dc8-3065-4707-93c0-7ef6a3373096 None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/98a63d5b605a4ecfad2e305c491ae755/roles/fc63a4da31e84ad78f04c587c0c1c83c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.216286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c025dc8-3065-4707-93c0-7ef6a3373096 None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=98a63d5b605a4ecfad2e305c491ae755, role_id=fc63a4da31e84ad78f04c587c0c1c83c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:12.218114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c025dc8-3065-4707-93c0-7ef6a3373096 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:12.232875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 279/564] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:10:12 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/98a63d5b605a4ecfad2e305c491ae755/roles/fc63a4da31e84ad78f04c587c0c1c83c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:10:12.238583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d264bf-2040-4aa6-913a-571c1a55609c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.238756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d264bf-2040-4aa6-913a-571c1a55609c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.238985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d264bf-2040-4aa6-913a-571c1a55609c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.246625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 280/565] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:12 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:12.248054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83063c16-6d34-46f5-be19-7e7cd0206875 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.248343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83063c16-6d34-46f5-be19-7e7cd0206875 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.250241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83063c16-6d34-46f5-be19-7e7cd0206875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.283641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-83063c16-6d34-46f5-be19-7e7cd0206875 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:12.286680 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d3fde-477a-4d03-88e9-f4d5e03ca23b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:12.288843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-83063c16-6d34-46f5-be19-7e7cd0206875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:12.348503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e61e9-e95a-4604-8243-144cb0fe00d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:12.349121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e61e9-e95a-4604-8243-144cb0fe00d6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.349395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e61e9-e95a-4604-8243-144cb0fe00d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.349663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e61e9-e95a-4604-8243-144cb0fe00d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.365333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:12.393860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 281/566] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:12.401493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83063c16-6d34-46f5-be19-7e7cd0206875 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:12.430059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e61e9-e95a-4604-8243-144cb0fe00d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:12.431547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e61e9-e95a-4604-8243-144cb0fe00d6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:12.519983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:12.528507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83063c16-6d34-46f5-be19-7e7cd0206875 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:12.549731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 286/567] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:10:12.551106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f1ba-636b-4642-8ea3-20523e6fdf16 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.551392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f1ba-636b-4642-8ea3-20523e6fdf16 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.551664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f1ba-636b-4642-8ea3-20523e6fdf16 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.553229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 287/568] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:12.561455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90f1ba-636b-4642-8ea3-20523e6fdf16 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:12.572465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce22dd-0cb7-4d3f-8159-8365d79cd42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:12.573740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce22dd-0cb7-4d3f-8159-8365d79cd42b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.574265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce22dd-0cb7-4d3f-8159-8365d79cd42b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.574739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce22dd-0cb7-4d3f-8159-8365d79cd42b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.587206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce22dd-0cb7-4d3f-8159-8365d79cd42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-adce22dd-0cb7-4d3f-8159-8365d79cd42b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:12.593257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:12.594261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:12.595188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:12.597946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 282/569] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:12.599490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d7e29-8345-4ae4-9409-969faed77458 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.600047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d7e29-8345-4ae4-9409-969faed77458 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.600624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d7e29-8345-4ae4-9409-969faed77458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.611913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:12.625947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-960d7e29-8345-4ae4-9409-969faed77458 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:12.627670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-960d7e29-8345-4ae4-9409-969faed77458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:12.686122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90f1ba-636b-4642-8ea3-20523e6fdf16 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:12.718513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 283/570] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:12.719746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d7e29-8345-4ae4-9409-969faed77458 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:12.732853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59221f08-d149-480a-a7e7-9ee88a0d057a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:12.734675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59221f08-d149-480a-a7e7-9ee88a0d057a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.735028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59221f08-d149-480a-a7e7-9ee88a0d057a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.735373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59221f08-d149-480a-a7e7-9ee88a0d057a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.752206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59221f08-d149-480a-a7e7-9ee88a0d057a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:12.754399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59221f08-d149-480a-a7e7-9ee88a0d057a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-59221f08-d149-480a-a7e7-9ee88a0d057a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:12.774519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:12.776716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:12.780107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:12.780107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:12.780107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:12.780107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 288/571] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:12.780107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59221f08-d149-480a-a7e7-9ee88a0d057a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:12.852847 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:12.878190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d7e29-8345-4ae4-9409-969faed77458 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:12.890963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:12.902895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59221f08-d149-480a-a7e7-9ee88a0d057a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:12.923939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5e795-8971-4314-b894-af2b066ccd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:12.924582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5e795-8971-4314-b894-af2b066ccd49 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.924863 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5e795-8971-4314-b894-af2b066ccd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.925556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5e795-8971-4314-b894-af2b066ccd49 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.926248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5e795-8971-4314-b894-af2b066ccd49 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:12.926553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5e795-8971-4314-b894-af2b066ccd49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:12.928202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5e795-8971-4314-b894-af2b066ccd49 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:12.939937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 284/572] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:12.940815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5e795-8971-4314-b894-af2b066ccd49 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:12.942192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b917f2-a522-4243-bf89-8591771a58b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:12.943149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b917f2-a522-4243-bf89-8591771a58b1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:12.943650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b917f2-a522-4243-bf89-8591771a58b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:12.944126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b917f2-a522-4243-bf89-8591771a58b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:12.956949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b917f2-a522-4243-bf89-8591771a58b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d9b917f2-a522-4243-bf89-8591771a58b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:12.962508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:12.963528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:12.964493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:12.967076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 289/573] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:12.968084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b917f2-a522-4243-bf89-8591771a58b1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.065324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5e795-8971-4314-b894-af2b066ccd49 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.085339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b917f2-a522-4243-bf89-8591771a58b1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.100065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561f9c9-dc91-4f81-9bbf-ce8c318bb0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.100699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561f9c9-dc91-4f81-9bbf-ce8c318bb0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.100968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561f9c9-dc91-4f81-9bbf-ce8c318bb0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.101246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561f9c9-dc91-4f81-9bbf-ce8c318bb0c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.101882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561f9c9-dc91-4f81-9bbf-ce8c318bb0c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:13.102181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561f9c9-dc91-4f81-9bbf-ce8c318bb0c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.103552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561f9c9-dc91-4f81-9bbf-ce8c318bb0c2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.121363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 285/574] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:13.122218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561f9c9-dc91-4f81-9bbf-ce8c318bb0c2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.122794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4841a-2d46-4357-9ee5-eb4b27a03e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.123430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4841a-2d46-4357-9ee5-eb4b27a03e92 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.123713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4841a-2d46-4357-9ee5-eb4b27a03e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.123990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4841a-2d46-4357-9ee5-eb4b27a03e92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.126607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:13.135507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4841a-2d46-4357-9ee5-eb4b27a03e92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.137184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4841a-2d46-4357-9ee5-eb4b27a03e92 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-71b4841a-2d46-4357-9ee5-eb4b27a03e92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:13.151649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:13.152630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:13.153537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:13.153537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:13.153537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:13.153537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 290/575] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:13.153537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4841a-2d46-4357-9ee5-eb4b27a03e92 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.164506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:13.230967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561f9c9-dc91-4f81-9bbf-ce8c318bb0c2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.263725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4841a-2d46-4357-9ee5-eb4b27a03e92 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.265550 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442357-a98f-4ca8-a8d0-3ec576bfaee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.266162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442357-a98f-4ca8-a8d0-3ec576bfaee3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.266445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442357-a98f-4ca8-a8d0-3ec576bfaee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.266713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442357-a98f-4ca8-a8d0-3ec576bfaee3 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.267382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87442357-a98f-4ca8-a8d0-3ec576bfaee3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:13.267683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87442357-a98f-4ca8-a8d0-3ec576bfaee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.269083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87442357-a98f-4ca8-a8d0-3ec576bfaee3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.283076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 286/576] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:13.283969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442357-a98f-4ca8-a8d0-3ec576bfaee3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.301355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbf550-dd52-4030-8535-81c9f198de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.302017 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbf550-dd52-4030-8535-81c9f198de8d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.302296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbf550-dd52-4030-8535-81c9f198de8d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.302571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbf550-dd52-4030-8535-81c9f198de8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.314089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbf550-dd52-4030-8535-81c9f198de8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-97fbf550-dd52-4030-8535-81c9f198de8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:13.319233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:13.320199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:13.321408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:13.325261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 291/577] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:13.326172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbf550-dd52-4030-8535-81c9f198de8d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.356594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:13.393230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442357-a98f-4ca8-a8d0-3ec576bfaee3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.404421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:13.430007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a547a5b-a5dc-419c-990d-606a60607fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.430635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a547a5b-a5dc-419c-990d-606a60607fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.430918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a547a5b-a5dc-419c-990d-606a60607fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.431284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a547a5b-a5dc-419c-990d-606a60607fb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.439150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a547a5b-a5dc-419c-990d-606a60607fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.440628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a547a5b-a5dc-419c-990d-606a60607fb2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.443438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbf550-dd52-4030-8535-81c9f198de8d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5a547a5b-a5dc-419c-990d-606a60607fb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:13.452726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:13.453641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:13.454552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:13.454552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:13.454552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:13.454552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:13.454552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:13.454552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:13.454552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:13.454552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:13.454552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 287/578] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:13.455038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a547a5b-a5dc-419c-990d-606a60607fb2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.484460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10149ca-c2bd-4cba-86a6-ca9fb1c114e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.485225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10149ca-c2bd-4cba-86a6-ca9fb1c114e8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.485524 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10149ca-c2bd-4cba-86a6-ca9fb1c114e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.485911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10149ca-c2bd-4cba-86a6-ca9fb1c114e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.486548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10149ca-c2bd-4cba-86a6-ca9fb1c114e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:13.486852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10149ca-c2bd-4cba-86a6-ca9fb1c114e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.488356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10149ca-c2bd-4cba-86a6-ca9fb1c114e8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.507175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 292/579] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:13.507175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10149ca-c2bd-4cba-86a6-ca9fb1c114e8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.569669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a547a5b-a5dc-419c-990d-606a60607fb2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.596209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:13.606433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c279a1-8147-400e-81e5-3e77d89fc6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.607062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c279a1-8147-400e-81e5-3e77d89fc6ea None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.607344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c279a1-8147-400e-81e5-3e77d89fc6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.607604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c279a1-8147-400e-81e5-3e77d89fc6ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.613908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10149ca-c2bd-4cba-86a6-ca9fb1c114e8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.618668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c279a1-8147-400e-81e5-3e77d89fc6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.620769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c279a1-8147-400e-81e5-3e77d89fc6ea None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-51c279a1-8147-400e-81e5-3e77d89fc6ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:13.634644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:13.636159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:13.637467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:13.637467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:13.637467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:13.637467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 288/580] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:13.637467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c279a1-8147-400e-81e5-3e77d89fc6ea None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.646426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:13.658350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7514-17db-4234-8888-d61e0a2dd7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.658956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7514-17db-4234-8888-d61e0a2dd7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.659237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7514-17db-4234-8888-d61e0a2dd7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.659513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca7514-17db-4234-8888-d61e0a2dd7f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.660175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca7514-17db-4234-8888-d61e0a2dd7f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:13.660506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca7514-17db-4234-8888-d61e0a2dd7f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.661949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca7514-17db-4234-8888-d61e0a2dd7f3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.674898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 293/581] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:13.675806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7514-17db-4234-8888-d61e0a2dd7f3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.754544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c279a1-8147-400e-81e5-3e77d89fc6ea None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.784631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca7514-17db-4234-8888-d61e0a2dd7f3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.794650 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a30ab-9e0d-4a5c-b8d0-3376c7fb78ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.795261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a30ab-9e0d-4a5c-b8d0-3376c7fb78ac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.795636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a30ab-9e0d-4a5c-b8d0-3376c7fb78ac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.795925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a30ab-9e0d-4a5c-b8d0-3376c7fb78ac None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.796652 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a30ab-9e0d-4a5c-b8d0-3376c7fb78ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:13.796961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a30ab-9e0d-4a5c-b8d0-3376c7fb78ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.798431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a30ab-9e0d-4a5c-b8d0-3376c7fb78ac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.812268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 289/582] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:13.813178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a30ab-9e0d-4a5c-b8d0-3376c7fb78ac None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.825979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28277-a7a5-422e-b925-fec4641a06a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.826543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28277-a7a5-422e-b925-fec4641a06a1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.826823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28277-a7a5-422e-b925-fec4641a06a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.827095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28277-a7a5-422e-b925-fec4641a06a1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.834883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e28277-a7a5-422e-b925-fec4641a06a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.836492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e28277-a7a5-422e-b925-fec4641a06a1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-91e28277-a7a5-422e-b925-fec4641a06a1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:13.847870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:13.848890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 294/583] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:13.850532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28277-a7a5-422e-b925-fec4641a06a1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:13.865697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:13.907268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:13.929093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a30ab-9e0d-4a5c-b8d0-3376c7fb78ac None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.964792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28277-a7a5-422e-b925-fec4641a06a1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:13.971631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b44005-90c7-47cb-abc9-be4537470926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:13.972240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b44005-90c7-47cb-abc9-be4537470926 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:13.972628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b44005-90c7-47cb-abc9-be4537470926 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:13.972891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b44005-90c7-47cb-abc9-be4537470926 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:13.973545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b44005-90c7-47cb-abc9-be4537470926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:13.973947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b44005-90c7-47cb-abc9-be4537470926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:13.975226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b44005-90c7-47cb-abc9-be4537470926 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:13.992870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 290/584] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:13.992870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b44005-90c7-47cb-abc9-be4537470926 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.017498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec472885-6ba0-403b-ae7b-596dc5cb5869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:14.018629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec472885-6ba0-403b-ae7b-596dc5cb5869 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.019348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec472885-6ba0-403b-ae7b-596dc5cb5869 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.019653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec472885-6ba0-403b-ae7b-596dc5cb5869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.035681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec472885-6ba0-403b-ae7b-596dc5cb5869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:14.037527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec472885-6ba0-403b-ae7b-596dc5cb5869 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ec472885-6ba0-403b-ae7b-596dc5cb5869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:14.056756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:14.059065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:14.061084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:14.061084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:14.061084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:14.064857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 295/585] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:14.065866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec472885-6ba0-403b-ae7b-596dc5cb5869 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.111911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:14.116498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b44005-90c7-47cb-abc9-be4537470926 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:14.160713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b197af-6f0b-4fc3-81b1-a34e223efda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:14.161319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b197af-6f0b-4fc3-81b1-a34e223efda0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.161600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b197af-6f0b-4fc3-81b1-a34e223efda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.161872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b197af-6f0b-4fc3-81b1-a34e223efda0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.162624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b197af-6f0b-4fc3-81b1-a34e223efda0 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=383ff491387642a3ab45ab692f94189c) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:14.162931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b197af-6f0b-4fc3-81b1-a34e223efda0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:14.164402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b197af-6f0b-4fc3-81b1-a34e223efda0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:14.171054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 291/586] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:13 2022] GET /identity/v3/role_assignments?user.id=383ff491387642a3ab45ab692f94189c&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:14.171875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b197af-6f0b-4fc3-81b1-a34e223efda0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.198674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:14.204543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec472885-6ba0-403b-ae7b-596dc5cb5869 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:14.257170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d45530-cbbb-44f7-b685-3754af9a02f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:14.258957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d45530-cbbb-44f7-b685-3754af9a02f3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.259679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d45530-cbbb-44f7-b685-3754af9a02f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.261009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d45530-cbbb-44f7-b685-3754af9a02f3 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.262456 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d45530-cbbb-44f7-b685-3754af9a02f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:14.263567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d45530-cbbb-44f7-b685-3754af9a02f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:14.266358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d45530-cbbb-44f7-b685-3754af9a02f3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:14.301410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 296/587] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Aug 19 15:10:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:14.302240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d45530-cbbb-44f7-b685-3754af9a02f3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.330468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b197af-6f0b-4fc3-81b1-a34e223efda0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:14.370807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb0836-ae39-41c0-833b-bf9dca2cee1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:14.371965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb0836-ae39-41c0-833b-bf9dca2cee1c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.372262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb0836-ae39-41c0-833b-bf9dca2cee1c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.372536 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb0836-ae39-41c0-833b-bf9dca2cee1c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.381659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb0836-ae39-41c0-833b-bf9dca2cee1c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:14.383143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb0836-ae39-41c0-833b-bf9dca2cee1c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-01fb0836-ae39-41c0-833b-bf9dca2cee1c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:14.395798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:14.397052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 15:10:14.397981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:14.400697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 292/588] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:14.402669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb0836-ae39-41c0-833b-bf9dca2cee1c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.428773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d45530-cbbb-44f7-b685-3754af9a02f3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:14.439104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:14.487077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77700e83-c1ba-43cf-a6cf-ec42e7efd67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:14.487698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77700e83-c1ba-43cf-a6cf-ec42e7efd67e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.487979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77700e83-c1ba-43cf-a6cf-ec42e7efd67e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.488275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77700e83-c1ba-43cf-a6cf-ec42e7efd67e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.488950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77700e83-c1ba-43cf-a6cf-ec42e7efd67e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:14.489255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77700e83-c1ba-43cf-a6cf-ec42e7efd67e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:14.490766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77700e83-c1ba-43cf-a6cf-ec42e7efd67e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:14.508858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 297/589] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:14.508858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246908b4-89d2-43d8-bbbd-a059716bf784 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.508858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246908b4-89d2-43d8-bbbd-a059716bf784 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.508858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246908b4-89d2-43d8-bbbd-a059716bf784 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.508858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 298/590] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:14.508858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246908b4-89d2-43d8-bbbd-a059716bf784 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.532646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:14.536554 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb0836-ae39-41c0-833b-bf9dca2cee1c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:14.575301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5ab90-d21a-4db4-8f8f-6430eb149799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:14.575930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5ab90-d21a-4db4-8f8f-6430eb149799 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.576205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5ab90-d21a-4db4-8f8f-6430eb149799 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.576511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5ab90-d21a-4db4-8f8f-6430eb149799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.587411 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5ab90-d21a-4db4-8f8f-6430eb149799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:14.588937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5ab90-d21a-4db4-8f8f-6430eb149799 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-91d5ab90-d21a-4db4-8f8f-6430eb149799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:14.602996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:14.604583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:14.605680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:14.605680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:14.605680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:14.605680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 293/591] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:14.605924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5ab90-d21a-4db4-8f8f-6430eb149799 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.630690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246908b4-89d2-43d8-bbbd-a059716bf784 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:14.675750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78349355-6c12-4b21-b1bc-310d2f07c257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:14.676608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78349355-6c12-4b21-b1bc-310d2f07c257 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.677050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78349355-6c12-4b21-b1bc-310d2f07c257 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.677329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78349355-6c12-4b21-b1bc-310d2f07c257 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.678073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78349355-6c12-4b21-b1bc-310d2f07c257 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:14.678384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78349355-6c12-4b21-b1bc-310d2f07c257 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:14.679852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78349355-6c12-4b21-b1bc-310d2f07c257 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:14.689924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:14.700359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 299/592] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Aug 19 15:10:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:14.702610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc93a8-ad21-456f-ad2f-7079270ce8e1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.703097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc93a8-ad21-456f-ad2f-7079270ce8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.703710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc93a8-ad21-456f-ad2f-7079270ce8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.724128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-93dc93a8-ad21-456f-ad2f-7079270ce8e1 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:14.724329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5ab90-d21a-4db4-8f8f-6430eb149799 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:14.725684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-93dc93a8-ad21-456f-ad2f-7079270ce8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:14.762466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ae074-3fa4-479d-9b42-083c6aa5373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:14.763115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ae074-3fa4-479d-9b42-083c6aa5373e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.763368 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ae074-3fa4-479d-9b42-083c6aa5373e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.763633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ae074-3fa4-479d-9b42-083c6aa5373e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.771852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ae074-3fa4-479d-9b42-083c6aa5373e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:14.773341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ae074-3fa4-479d-9b42-083c6aa5373e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2e6ae074-3fa4-479d-9b42-083c6aa5373e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:14.786768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:14.787754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:14.788743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 294/593] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:14.789330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ae074-3fa4-479d-9b42-083c6aa5373e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.805447 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 300/594] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:14.806628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc93a8-ad21-456f-ad2f-7079270ce8e1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.824200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:14.930638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ae074-3fa4-479d-9b42-083c6aa5373e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:14.952747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc93a8-ad21-456f-ad2f-7079270ce8e1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:14.970118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264533ad-47fa-42f8-b3d9-334efa24bf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:14.971203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264533ad-47fa-42f8-b3d9-334efa24bf63 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:14.971679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264533ad-47fa-42f8-b3d9-334efa24bf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:14.972180 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264533ad-47fa-42f8-b3d9-334efa24bf63 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:14.973178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264533ad-47fa-42f8-b3d9-334efa24bf63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:14.973629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264533ad-47fa-42f8-b3d9-334efa24bf63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:14.975306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264533ad-47fa-42f8-b3d9-334efa24bf63 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:14.977496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:14.991993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 295/595] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:14.993128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264533ad-47fa-42f8-b3d9-334efa24bf63 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:14.999040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb41ea-c9e6-4e74-bf94-ebc7167d1557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.000176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb41ea-c9e6-4e74-bf94-ebc7167d1557 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.001527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb41ea-c9e6-4e74-bf94-ebc7167d1557 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.001898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb41ea-c9e6-4e74-bf94-ebc7167d1557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.020222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb41ea-c9e6-4e74-bf94-ebc7167d1557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:15.022587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb41ea-c9e6-4e74-bf94-ebc7167d1557 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-24eb41ea-c9e6-4e74-bf94-ebc7167d1557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:15.047584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:15.048638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:15.049577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:15.049577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:15.049577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:15.052189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 301/596] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:15.053258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb41ea-c9e6-4e74-bf94-ebc7167d1557 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:15.125202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264533ad-47fa-42f8-b3d9-334efa24bf63 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.127887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:15.178559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba64b2-c2f9-4e93-be0e-3fe953111d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.179462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba64b2-c2f9-4e93-be0e-3fe953111d39 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.179991 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba64b2-c2f9-4e93-be0e-3fe953111d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.180615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba64b2-c2f9-4e93-be0e-3fe953111d39 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.182971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba64b2-c2f9-4e93-be0e-3fe953111d39 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:15.183385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba64b2-c2f9-4e93-be0e-3fe953111d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:15.184481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb41ea-c9e6-4e74-bf94-ebc7167d1557 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.186431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba64b2-c2f9-4e93-be0e-3fe953111d39 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:15.218187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 296/597] 10.222.0.75 () {56 vars in 1139 bytes} [Fri Aug 19 15:10:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:15.218187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba64b2-c2f9-4e93-be0e-3fe953111d39 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:15.231890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa690bfc-881d-42f7-9363-1605d89d5308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.232555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa690bfc-881d-42f7-9363-1605d89d5308 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.232845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa690bfc-881d-42f7-9363-1605d89d5308 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.233104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa690bfc-881d-42f7-9363-1605d89d5308 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.240924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa690bfc-881d-42f7-9363-1605d89d5308 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:15.242414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa690bfc-881d-42f7-9363-1605d89d5308 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-aa690bfc-881d-42f7-9363-1605d89d5308 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:15.254293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:15.255208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:15.256114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:15.256114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:15.256114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:15.256114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:15.256114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:15.256114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:15.256114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:15.256114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:15.256114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 302/598] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:15.256626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa690bfc-881d-42f7-9363-1605d89d5308 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:15.275157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:15.331743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba64b2-c2f9-4e93-be0e-3fe953111d39 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.369149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa690bfc-881d-42f7-9363-1605d89d5308 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.381747 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc629be4-d498-42a1-b3fe-17e6221f76d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.382389 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc629be4-d498-42a1-b3fe-17e6221f76d0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.382669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc629be4-d498-42a1-b3fe-17e6221f76d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.382940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc629be4-d498-42a1-b3fe-17e6221f76d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.383648 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc629be4-d498-42a1-b3fe-17e6221f76d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:15.383953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc629be4-d498-42a1-b3fe-17e6221f76d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:15.385407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc629be4-d498-42a1-b3fe-17e6221f76d0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:15.397336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:15.404020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 297/599] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:15.404020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc629be4-d498-42a1-b3fe-17e6221f76d0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:15.421239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445b239-b261-40da-8962-1c07ff813a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.421920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445b239-b261-40da-8962-1c07ff813a0a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.422196 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445b239-b261-40da-8962-1c07ff813a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.422465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445b239-b261-40da-8962-1c07ff813a0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.434111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445b239-b261-40da-8962-1c07ff813a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4445b239-b261-40da-8962-1c07ff813a0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:15.439416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:15.440339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:15.441216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 303/600] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:15.441893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445b239-b261-40da-8962-1c07ff813a0a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:15.533786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc629be4-d498-42a1-b3fe-17e6221f76d0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.550517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:15.555209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445b239-b261-40da-8962-1c07ff813a0a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.578780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668271dc-34ea-484e-a8bb-7831796aab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.579408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668271dc-34ea-484e-a8bb-7831796aab0a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.579691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668271dc-34ea-484e-a8bb-7831796aab0a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.579970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668271dc-34ea-484e-a8bb-7831796aab0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.592548 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668271dc-34ea-484e-a8bb-7831796aab0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:15.594016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668271dc-34ea-484e-a8bb-7831796aab0a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:15.597004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87896ebb-68f1-4145-af31-8e1741277a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.597615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87896ebb-68f1-4145-af31-8e1741277a92 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.597880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87896ebb-68f1-4145-af31-8e1741277a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.598147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87896ebb-68f1-4145-af31-8e1741277a92 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.598836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87896ebb-68f1-4145-af31-8e1741277a92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:15.599140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87896ebb-68f1-4145-af31-8e1741277a92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:15.600575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87896ebb-68f1-4145-af31-8e1741277a92 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-668271dc-34ea-484e-a8bb-7831796aab0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:15.610066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:15.611252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:15.612149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:15.612149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:15.612149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:15.612149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 298/601] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:15.612149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668271dc-34ea-484e-a8bb-7831796aab0a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:15.619265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 304/602] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:15.621355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87896ebb-68f1-4145-af31-8e1741277a92 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:15.669166 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:15.745327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668271dc-34ea-484e-a8bb-7831796aab0a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.761314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87896ebb-68f1-4145-af31-8e1741277a92 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.781940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7115bb0-8119-43e8-98d3-64ffc109d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.782574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7115bb0-8119-43e8-98d3-64ffc109d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.782852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7115bb0-8119-43e8-98d3-64ffc109d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.783123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7115bb0-8119-43e8-98d3-64ffc109d7b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.789932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7115bb0-8119-43e8-98d3-64ffc109d7b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:15.791388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7115bb0-8119-43e8-98d3-64ffc109d7b3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:15.800404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8f15e-382e-490d-8957-bee57199094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.801025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8f15e-382e-490d-8957-bee57199094d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.801308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8f15e-382e-490d-8957-bee57199094d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.801580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8f15e-382e-490d-8957-bee57199094d None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.802245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8f15e-382e-490d-8957-bee57199094d None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:15.802550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8f15e-382e-490d-8957-bee57199094d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-d7115bb0-8119-43e8-98d3-64ffc109d7b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:15.803251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:15.804107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:15.810817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:15.810817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:15.810817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:15.810817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:15.810817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:15.810817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:15.810817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:15.810817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:15.811261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8f15e-382e-490d-8957-bee57199094d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:15.811335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 299/603] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:15.811335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7115bb0-8119-43e8-98d3-64ffc109d7b3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:15.825307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 305/604] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:15.825307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8f15e-382e-490d-8957-bee57199094d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:15.834515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:15.926960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7115bb0-8119-43e8-98d3-64ffc109d7b3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.960954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:15.964763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8f15e-382e-490d-8957-bee57199094d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:15.968622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aede8-e875-443e-86c3-e2ada7597a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:15.969391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aede8-e875-443e-86c3-e2ada7597a35 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:15.969794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aede8-e875-443e-86c3-e2ada7597a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:15.970239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aede8-e875-443e-86c3-e2ada7597a35 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:15.971111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aede8-e875-443e-86c3-e2ada7597a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:15.971587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aede8-e875-443e-86c3-e2ada7597a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:15.973619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aede8-e875-443e-86c3-e2ada7597a35 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:15.993816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 300/605] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:15.993816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aede8-e875-443e-86c3-e2ada7597a35 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:16.015751 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b207a-27e1-4e01-82b5-3b4ba5d50c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:16.016536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b207a-27e1-4e01-82b5-3b4ba5d50c48 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.016880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b207a-27e1-4e01-82b5-3b4ba5d50c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.017220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b207a-27e1-4e01-82b5-3b4ba5d50c48 None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/de0dfcbad22348fdaea32d64557e68a6/roles/fc63a4da31e84ad78f04c587c0c1c83c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.034957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b207a-27e1-4e01-82b5-3b4ba5d50c48 None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=de0dfcbad22348fdaea32d64557e68a6, role_id=fc63a4da31e84ad78f04c587c0c1c83c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:16.036562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b207a-27e1-4e01-82b5-3b4ba5d50c48 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:16.052928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 306/606] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:10:15 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/de0dfcbad22348fdaea32d64557e68a6/roles/fc63a4da31e84ad78f04c587c0c1c83c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:10:16.065515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b207a-27e1-4e01-82b5-3b4ba5d50c48 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:16.105992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:16.125605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aede8-e875-443e-86c3-e2ada7597a35 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:16.168520 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5f970-7169-45f3-9ec0-cd72b7c38eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:16.169147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5f970-7169-45f3-9ec0-cd72b7c38eaa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.169430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5f970-7169-45f3-9ec0-cd72b7c38eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.169703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5f970-7169-45f3-9ec0-cd72b7c38eaa None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.170415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5f970-7169-45f3-9ec0-cd72b7c38eaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:16.170721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5f970-7169-45f3-9ec0-cd72b7c38eaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:16.172205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5f970-7169-45f3-9ec0-cd72b7c38eaa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:16.186265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 301/607] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:16.187134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5f970-7169-45f3-9ec0-cd72b7c38eaa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:16.220429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b207a-27e1-4e01-82b5-3b4ba5d50c48 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:16.261114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f593e0-a580-4d42-b236-54d3a53ed6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:16.261922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f593e0-a580-4d42-b236-54d3a53ed6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.262300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f593e0-a580-4d42-b236-54d3a53ed6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.262669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f593e0-a580-4d42-b236-54d3a53ed6e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.274504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f593e0-a580-4d42-b236-54d3a53ed6e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:16.276873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f593e0-a580-4d42-b236-54d3a53ed6e0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b8f593e0-a580-4d42-b236-54d3a53ed6e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:16.294760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:16.296032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:16.297117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:16.297117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:16.297117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:16.297117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 307/608] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:16.298254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a75e6-b064-4c9b-aa5b-99ee611591fe None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.298533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a75e6-b064-4c9b-aa5b-99ee611591fe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.298808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a75e6-b064-4c9b-aa5b-99ee611591fe None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.304157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 308/609] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:16.304157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a75e6-b064-4c9b-aa5b-99ee611591fe None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:16.307618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:16.317352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5f970-7169-45f3-9ec0-cd72b7c38eaa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:16.356575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9828034-9de6-4785-a692-da91df90cb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:16.357210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9828034-9de6-4785-a692-da91df90cb8d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.357490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9828034-9de6-4785-a692-da91df90cb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.357758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9828034-9de6-4785-a692-da91df90cb8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.365845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9828034-9de6-4785-a692-da91df90cb8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:16.367299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9828034-9de6-4785-a692-da91df90cb8d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a9828034-9de6-4785-a692-da91df90cb8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:16.379344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:16.380276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:16.381194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:16.381194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:16.381194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:16.381194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:16.381194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:16.381194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:16.381194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:16.381194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:16.381194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 302/610] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:16.385930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9828034-9de6-4785-a692-da91df90cb8d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:16.389507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:16.427962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a75e6-b064-4c9b-aa5b-99ee611591fe None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:16.461671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92145bac-17ab-4aaf-82a0-1a20957a339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:16.462326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92145bac-17ab-4aaf-82a0-1a20957a339a None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.462597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92145bac-17ab-4aaf-82a0-1a20957a339a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.462875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92145bac-17ab-4aaf-82a0-1a20957a339a None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/a6b4d6a93c7e49009aca527e281b5c3e/roles/fc63a4da31e84ad78f04c587c0c1c83c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.480646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92145bac-17ab-4aaf-82a0-1a20957a339a None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=a6b4d6a93c7e49009aca527e281b5c3e, role_id=fc63a4da31e84ad78f04c587c0c1c83c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:16.482091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92145bac-17ab-4aaf-82a0-1a20957a339a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:16.503992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 309/611] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:10:16 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/a6b4d6a93c7e49009aca527e281b5c3e/roles/fc63a4da31e84ad78f04c587c0c1c83c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:10:16.503992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0147-e436-4e85-b328-f27212c52b7e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.503992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0147-e436-4e85-b328-f27212c52b7e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.503992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0147-e436-4e85-b328-f27212c52b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.520442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9828034-9de6-4785-a692-da91df90cb8d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:16.533782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-961e0147-e436-4e85-b328-f27212c52b7e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:16.535869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-961e0147-e436-4e85-b328-f27212c52b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:16.553197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62461f88-7f2b-4651-9f01-eefc2148ebc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:16.553828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62461f88-7f2b-4651-9f01-eefc2148ebc7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.554189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62461f88-7f2b-4651-9f01-eefc2148ebc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.554468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62461f88-7f2b-4651-9f01-eefc2148ebc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.555357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62461f88-7f2b-4651-9f01-eefc2148ebc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:16.555664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62461f88-7f2b-4651-9f01-eefc2148ebc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:16.557399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62461f88-7f2b-4651-9f01-eefc2148ebc7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:16.571842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 303/612] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:16.572840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62461f88-7f2b-4651-9f01-eefc2148ebc7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:16.606094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:16.644416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 310/613] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:16.644416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0147-e436-4e85-b328-f27212c52b7e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:16.667739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:16.730694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62461f88-7f2b-4651-9f01-eefc2148ebc7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:16.767980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0147-e436-4e85-b328-f27212c52b7e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:16.770885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90d879-144f-45a3-bd62-97d76ff54da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:16.771608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90d879-144f-45a3-bd62-97d76ff54da6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.771988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90d879-144f-45a3-bd62-97d76ff54da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.772363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90d879-144f-45a3-bd62-97d76ff54da6 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.773183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90d879-144f-45a3-bd62-97d76ff54da6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:16.773590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90d879-144f-45a3-bd62-97d76ff54da6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:16.775195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90d879-144f-45a3-bd62-97d76ff54da6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:16.789104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 304/614] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:16.801524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90d879-144f-45a3-bd62-97d76ff54da6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:16.814219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941f969-6b26-4270-a8d0-e91c6ff4c480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:16.814855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941f969-6b26-4270-a8d0-e91c6ff4c480 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.815128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941f969-6b26-4270-a8d0-e91c6ff4c480 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.815396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941f969-6b26-4270-a8d0-e91c6ff4c480 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:16.829352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941f969-6b26-4270-a8d0-e91c6ff4c480 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:16.830830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941f969-6b26-4270-a8d0-e91c6ff4c480 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0941f969-6b26-4270-a8d0-e91c6ff4c480 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:16.851845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:16.852845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:16.853826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 311/615] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Aug 19 15:10:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:16.861345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941f969-6b26-4270-a8d0-e91c6ff4c480 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:16.944929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:16.947563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90d879-144f-45a3-bd62-97d76ff54da6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:16.972131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:16.984725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e70822-1dbe-4367-9465-191668eb5c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:16.985362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e70822-1dbe-4367-9465-191668eb5c92 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:16.985639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e70822-1dbe-4367-9465-191668eb5c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:16.985909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e70822-1dbe-4367-9465-191668eb5c92 None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/c8704aae85084d41bdedb5537ef96e44/roles/fc63a4da31e84ad78f04c587c0c1c83c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:17.003204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e70822-1dbe-4367-9465-191668eb5c92 None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=c8704aae85084d41bdedb5537ef96e44, role_id=fc63a4da31e84ad78f04c587c0c1c83c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:17.004935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e70822-1dbe-4367-9465-191668eb5c92 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:17.024457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 305/616] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:10:16 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/c8704aae85084d41bdedb5537ef96e44/roles/fc63a4da31e84ad78f04c587c0c1c83c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:10:17.024457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e70822-1dbe-4367-9465-191668eb5c92 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:17.040675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941f969-6b26-4270-a8d0-e91c6ff4c480 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:17.113589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa38f3-136c-41f4-9b11-bd14253667c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:17.114224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa38f3-136c-41f4-9b11-bd14253667c4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:17.114508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa38f3-136c-41f4-9b11-bd14253667c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:17.114784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa38f3-136c-41f4-9b11-bd14253667c4 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:17.115539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa38f3-136c-41f4-9b11-bd14253667c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:17.115844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa38f3-136c-41f4-9b11-bd14253667c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:17.117492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa38f3-136c-41f4-9b11-bd14253667c4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:17.140158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 312/617] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Aug 19 15:10:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:17.145846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa38f3-136c-41f4-9b11-bd14253667c4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:17.148656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e70822-1dbe-4367-9465-191668eb5c92 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:17.210861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb466c-b5ba-4863-97ea-a1cc1b05cdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:17.211491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb466c-b5ba-4863-97ea-a1cc1b05cdbb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:17.211772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb466c-b5ba-4863-97ea-a1cc1b05cdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:17.212082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb466c-b5ba-4863-97ea-a1cc1b05cdbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:17.224121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb466c-b5ba-4863-97ea-a1cc1b05cdbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-73fb466c-b5ba-4863-97ea-a1cc1b05cdbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:17.229972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:17.231009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:17.231921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 306/618] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:17.239513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb466c-b5ba-4863-97ea-a1cc1b05cdbb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:17.279538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:17.282627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa38f3-136c-41f4-9b11-bd14253667c4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:17.289135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:17.322421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48236f-a449-40cb-b596-ea869b7ae696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:17.323561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48236f-a449-40cb-b596-ea869b7ae696 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:17.323952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48236f-a449-40cb-b596-ea869b7ae696 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:17.324236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48236f-a449-40cb-b596-ea869b7ae696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:17.402176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48236f-a449-40cb-b596-ea869b7ae696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:17.403605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48236f-a449-40cb-b596-ea869b7ae696 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:17.439154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb466c-b5ba-4863-97ea-a1cc1b05cdbb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:17.505529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:17.510214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 313/619] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 15:10:17.516290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48236f-a449-40cb-b596-ea869b7ae696 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:17.525199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b7be-7534-4c22-9ae3-80e9c922f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:17.525850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b7be-7534-4c22-9ae3-80e9c922f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:17.526396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b7be-7534-4c22-9ae3-80e9c922f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:17.526672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b7be-7534-4c22-9ae3-80e9c922f5d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:17.527543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360b7be-7534-4c22-9ae3-80e9c922f5d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:17.528091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360b7be-7534-4c22-9ae3-80e9c922f5d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:17.529282 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360b7be-7534-4c22-9ae3-80e9c922f5d4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:17.547751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 307/620] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:17.554851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b7be-7534-4c22-9ae3-80e9c922f5d4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:17.696860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:17.706777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b7be-7534-4c22-9ae3-80e9c922f5d4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:17.713090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48236f-a449-40cb-b596-ea869b7ae696 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:17.771775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a319e1-768a-411b-9155-20b27d13ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:17.772873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a319e1-768a-411b-9155-20b27d13ae9b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:17.773384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a319e1-768a-411b-9155-20b27d13ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:17.773860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a319e1-768a-411b-9155-20b27d13ae9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:17.773969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972fc41-c7b8-4f27-9de9-77e94d9cb00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:17.774565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972fc41-c7b8-4f27-9de9-77e94d9cb00e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:17.774848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972fc41-c7b8-4f27-9de9-77e94d9cb00e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:17.775130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972fc41-c7b8-4f27-9de9-77e94d9cb00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:17.788424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a319e1-768a-411b-9155-20b27d13ae9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:17.790712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a319e1-768a-411b-9155-20b27d13ae9b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:17.792563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972fc41-c7b8-4f27-9de9-77e94d9cb00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:17.794025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972fc41-c7b8-4f27-9de9-77e94d9cb00e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3972fc41-c7b8-4f27-9de9-77e94d9cb00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:17.817167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:17.818186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:17.819088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:17.819088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:17.819088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-51a319e1-768a-411b-9155-20b27d13ae9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:17.824520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:17.825459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:17.826361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:17.826361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:17.826361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:17.826534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 308/621] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:17.827094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 314/622] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:17.831442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972fc41-c7b8-4f27-9de9-77e94d9cb00e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:17.832668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a319e1-768a-411b-9155-20b27d13ae9b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:17.947180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:17.994646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972fc41-c7b8-4f27-9de9-77e94d9cb00e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:18.033796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a319e1-768a-411b-9155-20b27d13ae9b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:18.062656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2f80-d1b1-432c-8692-0d0e0f783537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:18.063698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2f80-d1b1-432c-8692-0d0e0f783537 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.064205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2f80-d1b1-432c-8692-0d0e0f783537 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.064533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2f80-d1b1-432c-8692-0d0e0f783537 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.065869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c2f80-d1b1-432c-8692-0d0e0f783537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:18.065869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c2f80-d1b1-432c-8692-0d0e0f783537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:18.068008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c2f80-d1b1-432c-8692-0d0e0f783537 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:18.091985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 309/623] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:18.095870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a19707-75a4-4dd4-9d9a-4b6dae844147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:18.096506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a19707-75a4-4dd4-9d9a-4b6dae844147 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.096785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a19707-75a4-4dd4-9d9a-4b6dae844147 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.097056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a19707-75a4-4dd4-9d9a-4b6dae844147 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.097697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a19707-75a4-4dd4-9d9a-4b6dae844147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:18.097998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a19707-75a4-4dd4-9d9a-4b6dae844147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:18.099371 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a19707-75a4-4dd4-9d9a-4b6dae844147 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:18.101318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2f80-d1b1-432c-8692-0d0e0f783537 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:18.112717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:18.129362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 315/624] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:18.137715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a19707-75a4-4dd4-9d9a-4b6dae844147 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:18.234864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2f80-d1b1-432c-8692-0d0e0f783537 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:18.271749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5bbaf-6453-445f-901a-9056f5b13df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:18.272381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5bbaf-6453-445f-901a-9056f5b13df6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.272679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5bbaf-6453-445f-901a-9056f5b13df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.272952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5bbaf-6453-445f-901a-9056f5b13df6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.280854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a19707-75a4-4dd4-9d9a-4b6dae844147 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:18.281329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5bbaf-6453-445f-901a-9056f5b13df6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:18.282651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5bbaf-6453-445f-901a-9056f5b13df6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:18.295364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-96b5bbaf-6453-445f-901a-9056f5b13df6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:18.297522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:18.301128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 310/625] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f1f6-e1ae-403d-a1f5-a1ddae58dff4 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.302304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f1f6-e1ae-403d-a1f5-a1ddae58dff4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.303171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f1f6-e1ae-403d-a1f5-a1ddae58dff4 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.303171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 311/626] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:18.308108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674f1f6-e1ae-403d-a1f5-a1ddae58dff4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:18.326078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda9421-ee12-4af6-aca4-75b7e7aa6d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:18.326721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9421-ee12-4af6-aca4-75b7e7aa6d53 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.327014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9421-ee12-4af6-aca4-75b7e7aa6d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.327278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda9421-ee12-4af6-aca4-75b7e7aa6d53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.335678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda9421-ee12-4af6-aca4-75b7e7aa6d53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:18.337651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda9421-ee12-4af6-aca4-75b7e7aa6d53 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-feda9421-ee12-4af6-aca4-75b7e7aa6d53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:18.354768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:18.355889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:18.356932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:18.356932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:18.356932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:18.356932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:18.356932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:18.356932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:18.356932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:18.356932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:18.361331 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 316/627] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:18.363314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc02a6-1d17-409d-ab44-cc19c0fd7e8b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.363829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc02a6-1d17-409d-ab44-cc19c0fd7e8b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.366881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc02a6-1d17-409d-ab44-cc19c0fd7e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.387846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9fbc02a6-1d17-409d-ab44-cc19c0fd7e8b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:18.389095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9fbc02a6-1d17-409d-ab44-cc19c0fd7e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:18.414139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:18.464134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674f1f6-e1ae-403d-a1f5-a1ddae58dff4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:18.469430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 317/628] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:18.470537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc02a6-1d17-409d-ab44-cc19c0fd7e8b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:18.536375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8a9b-81bb-4f4a-9cd4-d9779aed4247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:18.537012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8a9b-81bb-4f4a-9cd4-d9779aed4247 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.537298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8a9b-81bb-4f4a-9cd4-d9779aed4247 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.539582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8a9b-81bb-4f4a-9cd4-d9779aed4247 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.540332 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d8a9b-81bb-4f4a-9cd4-d9779aed4247 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:18.540665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d8a9b-81bb-4f4a-9cd4-d9779aed4247 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:18.542137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d8a9b-81bb-4f4a-9cd4-d9779aed4247 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:18.586907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 312/629] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Aug 19 15:10:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:18.594138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8a9b-81bb-4f4a-9cd4-d9779aed4247 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:18.649489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc02a6-1d17-409d-ab44-cc19c0fd7e8b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:18.651714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:18.690146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ad749-de4e-41fb-9f9f-8f5d5b166b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:18.691020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad749-de4e-41fb-9f9f-8f5d5b166b4e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.691299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad749-de4e-41fb-9f9f-8f5d5b166b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.691576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad749-de4e-41fb-9f9f-8f5d5b166b4e None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.693230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ad749-de4e-41fb-9f9f-8f5d5b166b4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:18.693547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ad749-de4e-41fb-9f9f-8f5d5b166b4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:18.695009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ad749-de4e-41fb-9f9f-8f5d5b166b4e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:18.708584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 318/630] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:18.714562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ad749-de4e-41fb-9f9f-8f5d5b166b4e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:18.795289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8a9b-81bb-4f4a-9cd4-d9779aed4247 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:18.824640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:18.834229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63570854-44b7-4abf-816a-0b492946b532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:18.834949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63570854-44b7-4abf-816a-0b492946b532 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.835142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63570854-44b7-4abf-816a-0b492946b532 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.835380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63570854-44b7-4abf-816a-0b492946b532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.847359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63570854-44b7-4abf-816a-0b492946b532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:18.848860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63570854-44b7-4abf-816a-0b492946b532 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:18.854537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ad749-de4e-41fb-9f9f-8f5d5b166b4e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-63570854-44b7-4abf-816a-0b492946b532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:18.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:18.865185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:18.866129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:18.866129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:18.866129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:18.870528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 313/631] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:18.871541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63570854-44b7-4abf-816a-0b492946b532 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:18.903906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbbb69-06ed-4719-b7dc-36eaef17e2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:18.904568 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbbb69-06ed-4719-b7dc-36eaef17e2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:18.904845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbbb69-06ed-4719-b7dc-36eaef17e2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:18.905119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbbb69-06ed-4719-b7dc-36eaef17e2b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:18.905844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbbb69-06ed-4719-b7dc-36eaef17e2b1 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=98a63d5b605a4ecfad2e305c491ae755) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:18.906153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbbb69-06ed-4719-b7dc-36eaef17e2b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:18.907637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbbb69-06ed-4719-b7dc-36eaef17e2b1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:18.917386 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 319/632] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:18 2022] GET /identity/v3/role_assignments?user.id=98a63d5b605a4ecfad2e305c491ae755&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:18.917386 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbbb69-06ed-4719-b7dc-36eaef17e2b1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:18.949579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:19.004272 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63570854-44b7-4abf-816a-0b492946b532 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.049720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197ad21-b05a-48e6-a611-3f57b16b4f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:19.050343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197ad21-b05a-48e6-a611-3f57b16b4f1a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.050684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197ad21-b05a-48e6-a611-3f57b16b4f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.051009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197ad21-b05a-48e6-a611-3f57b16b4f1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.064861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197ad21-b05a-48e6-a611-3f57b16b4f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:19.067575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbbb69-06ed-4719-b7dc-36eaef17e2b1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5197ad21-b05a-48e6-a611-3f57b16b4f1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:19.070058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:19.071010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:19.071869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:19.076756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 314/633] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:19.078800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637aefa-0351-40ed-81b1-5ba3d68096c1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.079187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637aefa-0351-40ed-81b1-5ba3d68096c1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.079559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637aefa-0351-40ed-81b1-5ba3d68096c1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.083245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 315/634] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:19.084030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637aefa-0351-40ed-81b1-5ba3d68096c1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:19.113559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b47ee-83f6-45e0-97c7-79210b081f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:19.113782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b47ee-83f6-45e0-97c7-79210b081f0c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.114419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b47ee-83f6-45e0-97c7-79210b081f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.114419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b47ee-83f6-45e0-97c7-79210b081f0c None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.115025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b47ee-83f6-45e0-97c7-79210b081f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:19.115331 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b47ee-83f6-45e0-97c7-79210b081f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:19.117457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b47ee-83f6-45e0-97c7-79210b081f0c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:19.131316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 320/635] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:19.133213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127eeace-b75c-4657-aca4-f30d6d0c84d1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.133678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127eeace-b75c-4657-aca4-f30d6d0c84d1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.134049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127eeace-b75c-4657-aca4-f30d6d0c84d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.142423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:19.158095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-127eeace-b75c-4657-aca4-f30d6d0c84d1 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:19.158412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-127eeace-b75c-4657-aca4-f30d6d0c84d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:19.222776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637aefa-0351-40ed-81b1-5ba3d68096c1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.260585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 321/636] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:19.260585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127eeace-b75c-4657-aca4-f30d6d0c84d1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:19.266529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93662862-ad63-4c04-ae45-5f24a16ccbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:19.267858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93662862-ad63-4c04-ae45-5f24a16ccbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.268338 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93662862-ad63-4c04-ae45-5f24a16ccbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.268867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93662862-ad63-4c04-ae45-5f24a16ccbc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.269739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93662862-ad63-4c04-ae45-5f24a16ccbc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:19.270239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93662862-ad63-4c04-ae45-5f24a16ccbc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:19.272045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93662862-ad63-4c04-ae45-5f24a16ccbc5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:19.274197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:19.295259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 316/637] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:19.301437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93662862-ad63-4c04-ae45-5f24a16ccbc5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:19.378093 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127eeace-b75c-4657-aca4-f30d6d0c84d1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.423112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db80f9-46e4-4a0f-9a23-13f425f6cada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:19.423744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db80f9-46e4-4a0f-9a23-13f425f6cada None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.424025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db80f9-46e4-4a0f-9a23-13f425f6cada None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.424299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db80f9-46e4-4a0f-9a23-13f425f6cada None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.431663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:19.435746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93662862-ad63-4c04-ae45-5f24a16ccbc5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.436767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db80f9-46e4-4a0f-9a23-13f425f6cada None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:19.440767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db80f9-46e4-4a0f-9a23-13f425f6cada None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-94db80f9-46e4-4a0f-9a23-13f425f6cada None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:19.455674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:19.456683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:19.457667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:19.457667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:19.457667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:19.457667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:19.457667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:19.457667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:19.457667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:19.457667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:19.462225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 322/638] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:19.462225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0506f-f091-46bb-a2ce-fdd678ad0446 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.462225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0506f-f091-46bb-a2ce-fdd678ad0446 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.462225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0506f-f091-46bb-a2ce-fdd678ad0446 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.462225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 323/639] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:19.466796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0506f-f091-46bb-a2ce-fdd678ad0446 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:19.488874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befddd0d-2727-49c9-91bc-e8d813770799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:19.489939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befddd0d-2727-49c9-91bc-e8d813770799 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.490225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befddd0d-2727-49c9-91bc-e8d813770799 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.490502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befddd0d-2727-49c9-91bc-e8d813770799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.505467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befddd0d-2727-49c9-91bc-e8d813770799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:19.506978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befddd0d-2727-49c9-91bc-e8d813770799 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-befddd0d-2727-49c9-91bc-e8d813770799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:19.525378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:19.526856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:19.528535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:19.528535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:19.528535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:19.532103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 317/640] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:19.533709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff3f54-fb7e-4916-827d-3be582dbfa8e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.534000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff3f54-fb7e-4916-827d-3be582dbfa8e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.534275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff3f54-fb7e-4916-827d-3be582dbfa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.556066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-83ff3f54-fb7e-4916-827d-3be582dbfa8e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:19.558049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-83ff3f54-fb7e-4916-827d-3be582dbfa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:19.576969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:19.606445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0506f-f091-46bb-a2ce-fdd678ad0446 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.640419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 318/641] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:19.640419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff3f54-fb7e-4916-827d-3be582dbfa8e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:19.651540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e02b-5bcf-403b-85e0-61172dfe77e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:19.652224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e02b-5bcf-403b-85e0-61172dfe77e3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.652609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e02b-5bcf-403b-85e0-61172dfe77e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.652913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e02b-5bcf-403b-85e0-61172dfe77e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.653703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e02b-5bcf-403b-85e0-61172dfe77e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:19.654126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e02b-5bcf-403b-85e0-61172dfe77e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:19.655577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e02b-5bcf-403b-85e0-61172dfe77e3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:19.668251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 324/642] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:19.669249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e02b-5bcf-403b-85e0-61172dfe77e3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:19.738085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:19.766379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff3f54-fb7e-4916-827d-3be582dbfa8e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.791058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e02b-5bcf-403b-85e0-61172dfe77e3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.812162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca6868-d2f8-4215-a68d-93277d715592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:19.813138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca6868-d2f8-4215-a68d-93277d715592 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.813590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca6868-d2f8-4215-a68d-93277d715592 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.814066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca6868-d2f8-4215-a68d-93277d715592 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.815400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca6868-d2f8-4215-a68d-93277d715592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:19.816128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca6868-d2f8-4215-a68d-93277d715592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:19.818345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dca6868-d2f8-4215-a68d-93277d715592 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:19.829344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6512-211e-4de8-be71-41c5a2518b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:19.830106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6512-211e-4de8-be71-41c5a2518b20 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.830450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6512-211e-4de8-be71-41c5a2518b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.830797 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b6512-211e-4de8-be71-41c5a2518b20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:19.833921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:19.835273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 319/643] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:19.836277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca6868-d2f8-4215-a68d-93277d715592 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:19.848355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b6512-211e-4de8-be71-41c5a2518b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-677b6512-211e-4de8-be71-41c5a2518b20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:19.854802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:19.855876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:19.856743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:19.861087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 325/644] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:19.862083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6512-211e-4de8-be71-41c5a2518b20 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:19.955822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca6868-d2f8-4215-a68d-93277d715592 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.977259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b6512-211e-4de8-be71-41c5a2518b20 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:19.990617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca4d08-5373-4c6a-a377-f5ad94ec34b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:19.991274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca4d08-5373-4c6a-a377-f5ad94ec34b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:19.991555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca4d08-5373-4c6a-a377-f5ad94ec34b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:19.991834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca4d08-5373-4c6a-a377-f5ad94ec34b3 None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/383ff491387642a3ab45ab692f94189c/roles/fc63a4da31e84ad78f04c587c0c1c83c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.003848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:20.007724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca4d08-5373-4c6a-a377-f5ad94ec34b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=383ff491387642a3ab45ab692f94189c, role_id=fc63a4da31e84ad78f04c587c0c1c83c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.009209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca4d08-5373-4c6a-a377-f5ad94ec34b3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.013369 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86812d2a-068a-4462-a91b-5c9587d232fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.014055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86812d2a-068a-4462-a91b-5c9587d232fa None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.014328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86812d2a-068a-4462-a91b-5c9587d232fa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.014625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86812d2a-068a-4462-a91b-5c9587d232fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.022881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86812d2a-068a-4462-a91b-5c9587d232fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.032683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 320/645] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:10:19 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/383ff491387642a3ab45ab692f94189c/roles/fc63a4da31e84ad78f04c587c0c1c83c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:10:20.032895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86812d2a-068a-4462-a91b-5c9587d232fa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.033021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca4d08-5373-4c6a-a377-f5ad94ec34b3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-86812d2a-068a-4462-a91b-5c9587d232fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:20.048805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:20.050041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 19 15:10:20.051372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:20.052101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 326/646] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:20.055969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86812d2a-068a-4462-a91b-5c9587d232fa None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.137628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:20.169567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca4d08-5373-4c6a-a377-f5ad94ec34b3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:20.191619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86812d2a-068a-4462-a91b-5c9587d232fa None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:20.215326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa811e5-52ad-4ca2-b8e2-8a3cf0f210bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.215832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa811e5-52ad-4ca2-b8e2-8a3cf0f210bd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.216115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa811e5-52ad-4ca2-b8e2-8a3cf0f210bd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.216442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa811e5-52ad-4ca2-b8e2-8a3cf0f210bd None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.217067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa811e5-52ad-4ca2-b8e2-8a3cf0f210bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:20.217366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa811e5-52ad-4ca2-b8e2-8a3cf0f210bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.218769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa811e5-52ad-4ca2-b8e2-8a3cf0f210bd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.229287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 321/647] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:20.230079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa811e5-52ad-4ca2-b8e2-8a3cf0f210bd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.240008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31051f7-0efa-4bb7-87ad-40c278080536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.240814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31051f7-0efa-4bb7-87ad-40c278080536 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.241197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31051f7-0efa-4bb7-87ad-40c278080536 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.241570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31051f7-0efa-4bb7-87ad-40c278080536 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.256858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31051f7-0efa-4bb7-87ad-40c278080536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f31051f7-0efa-4bb7-87ad-40c278080536 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:20.265349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:20.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:20.267462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:20.269901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 327/648] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:20.271154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31051f7-0efa-4bb7-87ad-40c278080536 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.298410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:20.347369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa811e5-52ad-4ca2-b8e2-8a3cf0f210bd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:20.383408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbee7e-6733-4181-85df-2beebfa42965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.384175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbee7e-6733-4181-85df-2beebfa42965 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.384584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbee7e-6733-4181-85df-2beebfa42965 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.384959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbee7e-6733-4181-85df-2beebfa42965 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.385557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31051f7-0efa-4bb7-87ad-40c278080536 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:20.386711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbee7e-6733-4181-85df-2beebfa42965 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:20.386908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbee7e-6733-4181-85df-2beebfa42965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.388558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbee7e-6733-4181-85df-2beebfa42965 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.407393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:20.412034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 322/649] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:20.413167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbee7e-6733-4181-85df-2beebfa42965 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.428137 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012435b9-3565-4095-b946-3367d9b5f7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.429038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012435b9-3565-4095-b946-3367d9b5f7da None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.429462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012435b9-3565-4095-b946-3367d9b5f7da None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.429875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012435b9-3565-4095-b946-3367d9b5f7da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.443127 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012435b9-3565-4095-b946-3367d9b5f7da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.445299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012435b9-3565-4095-b946-3367d9b5f7da None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-012435b9-3565-4095-b946-3367d9b5f7da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:20.460633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:20.461824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:20.462854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:20.462854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:20.462854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:20.464643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 328/650] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:20.465698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012435b9-3565-4095-b946-3367d9b5f7da None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.528752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbee7e-6733-4181-85df-2beebfa42965 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:20.546030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:20.569579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700701f-f9bf-4457-a11f-5736bee9b9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.570271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700701f-f9bf-4457-a11f-5736bee9b9de None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.570658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700701f-f9bf-4457-a11f-5736bee9b9de None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.571035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700701f-f9bf-4457-a11f-5736bee9b9de None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.571812 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700701f-f9bf-4457-a11f-5736bee9b9de None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:20.572217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700701f-f9bf-4457-a11f-5736bee9b9de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.573907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700701f-f9bf-4457-a11f-5736bee9b9de None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.578892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012435b9-3565-4095-b946-3367d9b5f7da None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:20.589757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 323/651] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:20.589757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700701f-f9bf-4457-a11f-5736bee9b9de None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.621699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba78491-369a-4a19-a1bd-47fe377bf507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.622566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba78491-369a-4a19-a1bd-47fe377bf507 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.623023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba78491-369a-4a19-a1bd-47fe377bf507 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.623439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba78491-369a-4a19-a1bd-47fe377bf507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.660630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:20.694161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700701f-f9bf-4457-a11f-5736bee9b9de None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:20.700289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba78491-369a-4a19-a1bd-47fe377bf507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.701970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba78491-369a-4a19-a1bd-47fe377bf507 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.730658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa23b8-2bf2-403c-82c8-4e27907ff9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.731428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa23b8-2bf2-403c-82c8-4e27907ff9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.731807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa23b8-2bf2-403c-82c8-4e27907ff9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.732184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa23b8-2bf2-403c-82c8-4e27907ff9e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.732967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa23b8-2bf2-403c-82c8-4e27907ff9e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:20.733372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa23b8-2bf2-403c-82c8-4e27907ff9e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.734854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa23b8-2bf2-403c-82c8-4e27907ff9e0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.752788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 324/652] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:20.753778 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa23b8-2bf2-403c-82c8-4e27907ff9e0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.789066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:20.812081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 329/653] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:10:20.813523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba78491-369a-4a19-a1bd-47fe377bf507 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.865439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa23b8-2bf2-403c-82c8-4e27907ff9e0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:20.882697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:20.902804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510a67b-2a0e-48d4-81d5-981e7238e613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.903557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510a67b-2a0e-48d4-81d5-981e7238e613 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.903958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510a67b-2a0e-48d4-81d5-981e7238e613 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.904335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510a67b-2a0e-48d4-81d5-981e7238e613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.915176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510a67b-2a0e-48d4-81d5-981e7238e613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.917424 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510a67b-2a0e-48d4-81d5-981e7238e613 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.922865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba78491-369a-4a19-a1bd-47fe377bf507 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5510a67b-2a0e-48d4-81d5-981e7238e613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:20.934083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:20.935285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:20.936379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:20.936379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:20.936379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:20.942017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 325/654] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:20.942017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510a67b-2a0e-48d4-81d5-981e7238e613 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:20.968993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92587f40-b2a1-42d2-a71d-91858e05d110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:20.970281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92587f40-b2a1-42d2-a71d-91858e05d110 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:20.971046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92587f40-b2a1-42d2-a71d-91858e05d110 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:20.971573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92587f40-b2a1-42d2-a71d-91858e05d110 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:20.980609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92587f40-b2a1-42d2-a71d-91858e05d110 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:20.982296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92587f40-b2a1-42d2-a71d-91858e05d110 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-92587f40-b2a1-42d2-a71d-91858e05d110 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:20.994558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:20.995683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 15:10:20.996690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:20.999081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 330/655] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:21.000072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92587f40-b2a1-42d2-a71d-91858e05d110 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:21.036644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:21.053939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510a67b-2a0e-48d4-81d5-981e7238e613 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.087203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df55ad-a8b2-47ff-873a-18ab8d475814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.087826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df55ad-a8b2-47ff-873a-18ab8d475814 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.088111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df55ad-a8b2-47ff-873a-18ab8d475814 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.088988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df55ad-a8b2-47ff-873a-18ab8d475814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.099452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df55ad-a8b2-47ff-873a-18ab8d475814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:21.100963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df55ad-a8b2-47ff-873a-18ab8d475814 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:21.114370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92587f40-b2a1-42d2-a71d-91858e05d110 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b4df55ad-a8b2-47ff-873a-18ab8d475814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:21.115535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:21.116481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:21.117429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:21.117429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:21.117429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:21.117429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 326/656] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:21.117429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df55ad-a8b2-47ff-873a-18ab8d475814 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:21.149673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:21.161383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbba3b-442a-4226-896c-b6b3b7678848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.162010 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbba3b-442a-4226-896c-b6b3b7678848 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.162295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbba3b-442a-4226-896c-b6b3b7678848 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.162570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbba3b-442a-4226-896c-b6b3b7678848 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.163364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbba3b-442a-4226-896c-b6b3b7678848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:21.163671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbba3b-442a-4226-896c-b6b3b7678848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:21.165310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbba3b-442a-4226-896c-b6b3b7678848 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:21.183959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 331/657] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:21.185429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381885-e536-47e2-a57e-35fff81216fa None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.185716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381885-e536-47e2-a57e-35fff81216fa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.185997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381885-e536-47e2-a57e-35fff81216fa None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.191227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 332/658] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:21.191227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381885-e536-47e2-a57e-35fff81216fa None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:21.228409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df55ad-a8b2-47ff-873a-18ab8d475814 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.265900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676064c8-fc40-4167-b83b-5c0623babecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.266808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676064c8-fc40-4167-b83b-5c0623babecb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.267099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676064c8-fc40-4167-b83b-5c0623babecb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.267369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676064c8-fc40-4167-b83b-5c0623babecb None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.268095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676064c8-fc40-4167-b83b-5c0623babecb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:21.268416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676064c8-fc40-4167-b83b-5c0623babecb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:21.269863 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676064c8-fc40-4167-b83b-5c0623babecb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:21.279294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:21.290071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 327/659] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:21.290789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676064c8-fc40-4167-b83b-5c0623babecb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:21.302840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381885-e536-47e2-a57e-35fff81216fa None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.343086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ffaf2-bb54-4182-ba36-a20c259b0705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.344170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ffaf2-bb54-4182-ba36-a20c259b0705 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.344677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ffaf2-bb54-4182-ba36-a20c259b0705 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.345156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ffaf2-bb54-4182-ba36-a20c259b0705 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.345857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ffaf2-bb54-4182-ba36-a20c259b0705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:21.346162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ffaf2-bb54-4182-ba36-a20c259b0705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:21.347557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ffaf2-bb54-4182-ba36-a20c259b0705 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:21.360528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 333/660] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:21.362132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e1725-117c-48b8-9bb4-0edb4c9ce20f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.362674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e1725-117c-48b8-9bb4-0edb4c9ce20f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.363187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e1725-117c-48b8-9bb4-0edb4c9ce20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.382123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3e4e1725-117c-48b8-9bb4-0edb4c9ce20f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:21.383314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3e4e1725-117c-48b8-9bb4-0edb4c9ce20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:21.397454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676064c8-fc40-4167-b83b-5c0623babecb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.433654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:21.436758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597715b-5474-4dc0-a27b-b966716a65cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.437553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597715b-5474-4dc0-a27b-b966716a65cd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.437838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597715b-5474-4dc0-a27b-b966716a65cd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.438431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597715b-5474-4dc0-a27b-b966716a65cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.445959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597715b-5474-4dc0-a27b-b966716a65cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:21.447404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597715b-5474-4dc0-a27b-b966716a65cd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:21.464451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 334/661] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0597715b-5474-4dc0-a27b-b966716a65cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:21.464627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:21.466514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 15:10:21.467449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:21.467993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e1725-117c-48b8-9bb4-0edb4c9ce20f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:21.468070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 328/662] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:21.468070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597715b-5474-4dc0-a27b-b966716a65cd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:21.534141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:21.581224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e1725-117c-48b8-9bb4-0edb4c9ce20f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.582658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597715b-5474-4dc0-a27b-b966716a65cd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.617878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2af3a5-1a51-472d-8b39-7ae7770a901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.618590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2af3a5-1a51-472d-8b39-7ae7770a901c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.618866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2af3a5-1a51-472d-8b39-7ae7770a901c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.619225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2af3a5-1a51-472d-8b39-7ae7770a901c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.620907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7c6b3-6bb8-4001-baeb-52ce193ba30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.621641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7c6b3-6bb8-4001-baeb-52ce193ba30a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.622024 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7c6b3-6bb8-4001-baeb-52ce193ba30a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.622508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7c6b3-6bb8-4001-baeb-52ce193ba30a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.630021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7c6b3-6bb8-4001-baeb-52ce193ba30a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:21.631754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7c6b3-6bb8-4001-baeb-52ce193ba30a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f7c7c6b3-6bb8-4001-baeb-52ce193ba30a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:21.645954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:21.647066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:21.647979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:21.647979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:21.647979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:21.647979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:21.647979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:21.647979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:21.647979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:21.647979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:21.647979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 329/663] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:21.651118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7c6b3-6bb8-4001-baeb-52ce193ba30a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:21.678937 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:21.704995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2af3a5-1a51-472d-8b39-7ae7770a901c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:21.706907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2af3a5-1a51-472d-8b39-7ae7770a901c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:21.763286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7c6b3-6bb8-4001-baeb-52ce193ba30a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.781790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:21.800905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e81bd-23e2-4107-a41b-1a3590ad3c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.801595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e81bd-23e2-4107-a41b-1a3590ad3c34 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.801952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e81bd-23e2-4107-a41b-1a3590ad3c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.802301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e81bd-23e2-4107-a41b-1a3590ad3c34 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.803107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e81bd-23e2-4107-a41b-1a3590ad3c34 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:21.803485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e81bd-23e2-4107-a41b-1a3590ad3c34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:21.805028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e81bd-23e2-4107-a41b-1a3590ad3c34 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:21.821020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 335/664] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:10:21.822141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2af3a5-1a51-472d-8b39-7ae7770a901c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:21.828459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 330/665] 10.222.0.75 () {56 vars in 1139 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:21.829434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e81bd-23e2-4107-a41b-1a3590ad3c34 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:21.909834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:21.933632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e81bd-23e2-4107-a41b-1a3590ad3c34 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.937206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2af3a5-1a51-472d-8b39-7ae7770a901c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:21.971575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6661d-7acc-4b27-b4ed-d55af5a45566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.972327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6661d-7acc-4b27-b4ed-d55af5a45566 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.972745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6661d-7acc-4b27-b4ed-d55af5a45566 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.972916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6661d-7acc-4b27-b4ed-d55af5a45566 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.974757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb74a2-ab53-4e87-8cdc-4139198ec7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:21.975364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb74a2-ab53-4e87-8cdc-4139198ec7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:21.975645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb74a2-ab53-4e87-8cdc-4139198ec7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:21.976080 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb74a2-ab53-4e87-8cdc-4139198ec7a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:21.976704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb74a2-ab53-4e87-8cdc-4139198ec7a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:21.977018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb74a2-ab53-4e87-8cdc-4139198ec7a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:21.978464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb74a2-ab53-4e87-8cdc-4139198ec7a0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:21.984754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6661d-7acc-4b27-b4ed-d55af5a45566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-58d6661d-7acc-4b27-b4ed-d55af5a45566 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:21.989405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:21.990314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:21.991175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 331/666] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:21.991874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6661d-7acc-4b27-b4ed-d55af5a45566 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.002600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 336/667] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:22.002600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb74a2-ab53-4e87-8cdc-4139198ec7a0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.028619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:22.095601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6661d-7acc-4b27-b4ed-d55af5a45566 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.109727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb74a2-ab53-4e87-8cdc-4139198ec7a0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.128281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa595a00-6206-48a2-93da-c4ec89518895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:22.128922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa595a00-6206-48a2-93da-c4ec89518895 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.129203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa595a00-6206-48a2-93da-c4ec89518895 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.129465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa595a00-6206-48a2-93da-c4ec89518895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.139943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa595a00-6206-48a2-93da-c4ec89518895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:22.141429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa595a00-6206-48a2-93da-c4ec89518895 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:22.143346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d51c-adb9-4335-8aa2-a2c47d6ca1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:22.143953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d51c-adb9-4335-8aa2-a2c47d6ca1bc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.144232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d51c-adb9-4335-8aa2-a2c47d6ca1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.144537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3d51c-adb9-4335-8aa2-a2c47d6ca1bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.153754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-aa595a00-6206-48a2-93da-c4ec89518895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:22.156310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:22.157449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:22.158374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:22.158374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:22.158374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:22.158374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 332/668] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:22.158374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa595a00-6206-48a2-93da-c4ec89518895 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.218896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3d51c-adb9-4335-8aa2-a2c47d6ca1bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:22.220297 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3d51c-adb9-4335-8aa2-a2c47d6ca1bc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:22.240994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:22.256056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa595a00-6206-48a2-93da-c4ec89518895 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.290163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08b4e1-0d97-453d-8933-2f96cfc44b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:22.290758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08b4e1-0d97-453d-8933-2f96cfc44b73 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.291041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08b4e1-0d97-453d-8933-2f96cfc44b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.291408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08b4e1-0d97-453d-8933-2f96cfc44b73 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.292284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08b4e1-0d97-453d-8933-2f96cfc44b73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:22.292626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08b4e1-0d97-453d-8933-2f96cfc44b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:22.294001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08b4e1-0d97-453d-8933-2f96cfc44b73 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:22.302037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 333/669] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:22.303345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e562b6-b89f-485b-9e8a-eb564a7f5fb7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.303630 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e562b6-b89f-485b-9e8a-eb564a7f5fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.303902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e562b6-b89f-485b-9e8a-eb564a7f5fb7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.305100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 334/670] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:22.305816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e562b6-b89f-485b-9e8a-eb564a7f5fb7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.323858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 337/671] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:10:22.324695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d51c-adb9-4335-8aa2-a2c47d6ca1bc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.363605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:22.410721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e562b6-b89f-485b-9e8a-eb564a7f5fb7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.425438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3d51c-adb9-4335-8aa2-a2c47d6ca1bc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.443626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c06202-394b-43ee-ba15-739791d1dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:22.444216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c06202-394b-43ee-ba15-739791d1dd2b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.444527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c06202-394b-43ee-ba15-739791d1dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.444795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c06202-394b-43ee-ba15-739791d1dd2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.445514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c06202-394b-43ee-ba15-739791d1dd2b None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=de0dfcbad22348fdaea32d64557e68a6) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:22.445823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c06202-394b-43ee-ba15-739791d1dd2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:22.447259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c06202-394b-43ee-ba15-739791d1dd2b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:22.453828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 335/672] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:22 2022] GET /identity/v3/role_assignments?user.id=de0dfcbad22348fdaea32d64557e68a6&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:22.454613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c06202-394b-43ee-ba15-739791d1dd2b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.469842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:22.474660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688a3dc-a8d9-447f-b96d-de439983600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:22.475406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688a3dc-a8d9-447f-b96d-de439983600e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.475683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688a3dc-a8d9-447f-b96d-de439983600e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.475954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688a3dc-a8d9-447f-b96d-de439983600e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.476650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688a3dc-a8d9-447f-b96d-de439983600e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:22.476951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688a3dc-a8d9-447f-b96d-de439983600e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:22.478432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688a3dc-a8d9-447f-b96d-de439983600e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:22.491694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 338/673] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:22.493008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1e6a1-a9d9-4727-a9af-a3d61e72845f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.493347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1e6a1-a9d9-4727-a9af-a3d61e72845f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.493645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1e6a1-a9d9-4727-a9af-a3d61e72845f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.513825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-51f1e6a1-a9d9-4727-a9af-a3d61e72845f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:22.515179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-51f1e6a1-a9d9-4727-a9af-a3d61e72845f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:22.579523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c06202-394b-43ee-ba15-739791d1dd2b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.592448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 339/674] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:22.592448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1e6a1-a9d9-4727-a9af-a3d61e72845f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.618762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6653f-98e3-4274-b4e4-25c3f39437eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:22.618762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6653f-98e3-4274-b4e4-25c3f39437eb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.618762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6653f-98e3-4274-b4e4-25c3f39437eb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.618762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6653f-98e3-4274-b4e4-25c3f39437eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.629562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:22.632232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6653f-98e3-4274-b4e4-25c3f39437eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:22.633793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6653f-98e3-4274-b4e4-25c3f39437eb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-12e6653f-98e3-4274-b4e4-25c3f39437eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:22.653467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:22.654572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:22.655651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:22.655651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:22.655651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:22.656784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 336/675] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:22.658693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6653f-98e3-4274-b4e4-25c3f39437eb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.734319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1e6a1-a9d9-4727-a9af-a3d61e72845f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.737789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:22.780911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5e9c9-056f-49ed-9ca3-1cd0e6cc2264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:22.782393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5e9c9-056f-49ed-9ca3-1cd0e6cc2264 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.782917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5e9c9-056f-49ed-9ca3-1cd0e6cc2264 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.783334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc5e9c9-056f-49ed-9ca3-1cd0e6cc2264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.784622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6653f-98e3-4274-b4e4-25c3f39437eb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.797416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5e9c9-056f-49ed-9ca3-1cd0e6cc2264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:22.801611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc5e9c9-056f-49ed-9ca3-1cd0e6cc2264 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-cbc5e9c9-056f-49ed-9ca3-1cd0e6cc2264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:22.821564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:22.823360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:22.824460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:22.824460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:22.824460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:22.826232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 340/676] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:22.831238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5e9c9-056f-49ed-9ca3-1cd0e6cc2264 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.834445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6ef91-bba9-4c7b-a5ff-f2f86662004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:22.835237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6ef91-bba9-4c7b-a5ff-f2f86662004d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.835646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6ef91-bba9-4c7b-a5ff-f2f86662004d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.836073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6ef91-bba9-4c7b-a5ff-f2f86662004d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.845340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6ef91-bba9-4c7b-a5ff-f2f86662004d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:22.847030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6ef91-bba9-4c7b-a5ff-f2f86662004d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-99a6ef91-bba9-4c7b-a5ff-f2f86662004d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:22.859074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:22.860132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:22.861239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:22.861239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:22.861239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:22.861239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:22.861239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:22.861239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:22.861239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:22.861239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:22.863307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 337/677] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:22.864299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6ef91-bba9-4c7b-a5ff-f2f86662004d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:22.929938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:22.955800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc5e9c9-056f-49ed-9ca3-1cd0e6cc2264 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.973706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6ef91-bba9-4c7b-a5ff-f2f86662004d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:22.992755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadae46-6d81-47a7-b7e3-8fdbbb0f94c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:22.993283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadae46-6d81-47a7-b7e3-8fdbbb0f94c3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:22.993561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadae46-6d81-47a7-b7e3-8fdbbb0f94c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:22.993831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadae46-6d81-47a7-b7e3-8fdbbb0f94c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:22.994511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadae46-6d81-47a7-b7e3-8fdbbb0f94c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:22.995353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadae46-6d81-47a7-b7e3-8fdbbb0f94c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:22.996653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aadae46-6d81-47a7-b7e3-8fdbbb0f94c3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:22.999411 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:23.014138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 341/678] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:23.014138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadae46-6d81-47a7-b7e3-8fdbbb0f94c3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.016026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674aa30-2253-4640-97c4-7d1246590e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.016745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674aa30-2253-4640-97c4-7d1246590e24 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.017100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674aa30-2253-4640-97c4-7d1246590e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.017449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674aa30-2253-4640-97c4-7d1246590e24 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.018195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674aa30-2253-4640-97c4-7d1246590e24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:23.018568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674aa30-2253-4640-97c4-7d1246590e24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:23.020141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7674aa30-2253-4640-97c4-7d1246590e24 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:23.037900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 338/679] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:23.037900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674aa30-2253-4640-97c4-7d1246590e24 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.129750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadae46-6d81-47a7-b7e3-8fdbbb0f94c3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.145792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7674aa30-2253-4640-97c4-7d1246590e24 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.166536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efee87-2850-411b-b116-864e75cc4809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.167171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efee87-2850-411b-b116-864e75cc4809 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.167454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efee87-2850-411b-b116-864e75cc4809 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.168011 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efee87-2850-411b-b116-864e75cc4809 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.168451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efee87-2850-411b-b116-864e75cc4809 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:23.168764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efee87-2850-411b-b116-864e75cc4809 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:23.170222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6efee87-2850-411b-b116-864e75cc4809 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:23.179220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:23.183223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf9367-6104-452a-b67a-60f2bf08a7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.183849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf9367-6104-452a-b67a-60f2bf08a7ee None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.184134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf9367-6104-452a-b67a-60f2bf08a7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.184414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf9367-6104-452a-b67a-60f2bf08a7ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.188436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 342/680] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:23.189256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efee87-2850-411b-b116-864e75cc4809 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.194900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf9367-6104-452a-b67a-60f2bf08a7ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-cacf9367-6104-452a-b67a-60f2bf08a7ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:23.200207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:23.201194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 339/681] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:23.202045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf9367-6104-452a-b67a-60f2bf08a7ee None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.252071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:23.309256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efee87-2850-411b-b116-864e75cc4809 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.312317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf9367-6104-452a-b67a-60f2bf08a7ee None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.347416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db613043-df7c-447b-bb88-914b477cbf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.348043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db613043-df7c-447b-bb88-914b477cbf59 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.348348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db613043-df7c-447b-bb88-914b477cbf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.348642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db613043-df7c-447b-bb88-914b477cbf59 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.350582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e1412-f0c1-44ad-931f-5e586cbb95aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.351517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e1412-f0c1-44ad-931f-5e586cbb95aa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.351802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e1412-f0c1-44ad-931f-5e586cbb95aa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.352076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e1412-f0c1-44ad-931f-5e586cbb95aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.356843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db613043-df7c-447b-bb88-914b477cbf59 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:23.358306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db613043-df7c-447b-bb88-914b477cbf59 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:23.359902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e1412-f0c1-44ad-931f-5e586cbb95aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:23.361398 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e1412-f0c1-44ad-931f-5e586cbb95aa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-db613043-df7c-447b-bb88-914b477cbf59 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:23.369658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:23.370567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:23.371488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-c44e1412-f0c1-44ad-931f-5e586cbb95aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:23.372839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:23.373667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:23.374539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:23.374539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:23.374539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:23.374539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:23.374539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:23.374539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:23.374539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:23.374539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:23.375672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 343/682] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:23.376419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db613043-df7c-447b-bb88-914b477cbf59 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.378542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 340/683] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:23.379357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e1412-f0c1-44ad-931f-5e586cbb95aa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.444555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:23.490595 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db613043-df7c-447b-bb88-914b477cbf59 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.494360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e1412-f0c1-44ad-931f-5e586cbb95aa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.497625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:23.531109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bca0af-2748-4a8b-a431-53b511efcbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.531804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bca0af-2748-4a8b-a431-53b511efcbbb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.532079 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bca0af-2748-4a8b-a431-53b511efcbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.532350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bca0af-2748-4a8b-a431-53b511efcbbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.533094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bca0af-2748-4a8b-a431-53b511efcbbb None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=a6b4d6a93c7e49009aca527e281b5c3e) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:23.533398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bca0af-2748-4a8b-a431-53b511efcbbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:23.534131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ff380-c951-4150-a113-374606530d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.534830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bca0af-2748-4a8b-a431-53b511efcbbb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:23.534924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ff380-c951-4150-a113-374606530d52 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.535279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ff380-c951-4150-a113-374606530d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.535651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ff380-c951-4150-a113-374606530d52 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.536500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ff380-c951-4150-a113-374606530d52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:23.536821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ff380-c951-4150-a113-374606530d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:23.538294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ff380-c951-4150-a113-374606530d52 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:23.548670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 344/684] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/role_assignments?user.id=a6b4d6a93c7e49009aca527e281b5c3e&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:23.548670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bca0af-2748-4a8b-a431-53b511efcbbb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.554983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 341/685] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:23.556524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ff380-c951-4150-a113-374606530d52 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.679425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bca0af-2748-4a8b-a431-53b511efcbbb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.707992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:23.710671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ff380-c951-4150-a113-374606530d52 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.719248 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956cb50-864e-4423-88a9-f4a345de6093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.719859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956cb50-864e-4423-88a9-f4a345de6093 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.720203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956cb50-864e-4423-88a9-f4a345de6093 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.720516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956cb50-864e-4423-88a9-f4a345de6093 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.721257 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956cb50-864e-4423-88a9-f4a345de6093 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:23.721565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956cb50-864e-4423-88a9-f4a345de6093 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:23.723005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956cb50-864e-4423-88a9-f4a345de6093 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:23.741635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 345/686] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:23.742520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956cb50-864e-4423-88a9-f4a345de6093 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.752046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099f0eb-215d-4fd3-a32f-acf58e96bfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.752698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099f0eb-215d-4fd3-a32f-acf58e96bfab None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.752973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099f0eb-215d-4fd3-a32f-acf58e96bfab None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.753240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099f0eb-215d-4fd3-a32f-acf58e96bfab None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.753941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099f0eb-215d-4fd3-a32f-acf58e96bfab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:23.754239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099f0eb-215d-4fd3-a32f-acf58e96bfab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:23.755891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099f0eb-215d-4fd3-a32f-acf58e96bfab None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:23.770711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 342/687] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:23.771382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099f0eb-215d-4fd3-a32f-acf58e96bfab None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.809227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:23.856086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956cb50-864e-4423-88a9-f4a345de6093 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.888467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099f0eb-215d-4fd3-a32f-acf58e96bfab None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:23.892341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8cb4b-2203-4848-a86a-73d360cde6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.893074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8cb4b-2203-4848-a86a-73d360cde6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.893352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8cb4b-2203-4848-a86a-73d360cde6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.893623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8cb4b-2203-4848-a86a-73d360cde6e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.904585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8cb4b-2203-4848-a86a-73d360cde6e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:23.906051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8cb4b-2203-4848-a86a-73d360cde6e6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-00b8cb4b-2203-4848-a86a-73d360cde6e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:23.923289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:23.924303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:23.925308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:23.925308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:23.925308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:23.925308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 346/688] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:23.926528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8cb4b-2203-4848-a86a-73d360cde6e6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:23.930791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b79e3-b724-448d-8dc5-daac577afebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:23.931412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b79e3-b724-448d-8dc5-daac577afebb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:23.931756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b79e3-b724-448d-8dc5-daac577afebb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:23.932656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b79e3-b724-448d-8dc5-daac577afebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:23.964499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:24.007333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b79e3-b724-448d-8dc5-daac577afebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:24.008837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b79e3-b724-448d-8dc5-daac577afebb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:24.035498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8cb4b-2203-4848-a86a-73d360cde6e6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:24.038430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:24.074760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de738e-8f1e-4559-b53d-a91d102b3864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:24.075296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de738e-8f1e-4559-b53d-a91d102b3864 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.075615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de738e-8f1e-4559-b53d-a91d102b3864 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.075958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de738e-8f1e-4559-b53d-a91d102b3864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:24.083671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de738e-8f1e-4559-b53d-a91d102b3864 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=c8704aae85084d41bdedb5537ef96e44) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:24.083671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de738e-8f1e-4559-b53d-a91d102b3864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:24.083671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de738e-8f1e-4559-b53d-a91d102b3864 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:24.092965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 347/689] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/role_assignments?user.id=c8704aae85084d41bdedb5537ef96e44&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:24.092965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de738e-8f1e-4559-b53d-a91d102b3864 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:24.177674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 343/690] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:10:24.183733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b79e3-b724-448d-8dc5-daac577afebb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:24.249843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:24.259281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de738e-8f1e-4559-b53d-a91d102b3864 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:24.325927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b79e3-b724-448d-8dc5-daac577afebb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:24.332015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf1f0d-a559-48e7-a763-d27ceb59aac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:24.332770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf1f0d-a559-48e7-a763-d27ceb59aac0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.333146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf1f0d-a559-48e7-a763-d27ceb59aac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.333501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf1f0d-a559-48e7-a763-d27ceb59aac0 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:24.334246 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf1f0d-a559-48e7-a763-d27ceb59aac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:24.334952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf1f0d-a559-48e7-a763-d27ceb59aac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:24.336234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf1f0d-a559-48e7-a763-d27ceb59aac0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:24.356152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:24.358400 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 348/691] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:24.365421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf1f0d-a559-48e7-a763-d27ceb59aac0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:24.374350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53f1f8-86b6-4757-a928-f1a22f87294b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:24.374982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53f1f8-86b6-4757-a928-f1a22f87294b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.375257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53f1f8-86b6-4757-a928-f1a22f87294b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.375524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53f1f8-86b6-4757-a928-f1a22f87294b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:24.388082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53f1f8-86b6-4757-a928-f1a22f87294b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:24.389604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53f1f8-86b6-4757-a928-f1a22f87294b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ea53f1f8-86b6-4757-a928-f1a22f87294b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:24.409131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:24.410149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:24.411074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:24.411074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:24.411074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:24.415066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 344/692] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:24.417701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53f1f8-86b6-4757-a928-f1a22f87294b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:24.495368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf1f0d-a559-48e7-a763-d27ceb59aac0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:24.529440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53f1f8-86b6-4757-a928-f1a22f87294b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:24.533902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14cb84-6949-4d87-845f-9250fd2502eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:24.534809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14cb84-6949-4d87-845f-9250fd2502eb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.535398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14cb84-6949-4d87-845f-9250fd2502eb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.535902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14cb84-6949-4d87-845f-9250fd2502eb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:24.544611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14cb84-6949-4d87-845f-9250fd2502eb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:24.546427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14cb84-6949-4d87-845f-9250fd2502eb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-df14cb84-6949-4d87-845f-9250fd2502eb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:24.559314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:24.560300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:24.561300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:24.563807 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 349/693] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:24.569548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14cb84-6949-4d87-845f-9250fd2502eb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:24.575397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f2900-567f-4a1f-aae3-12bd0696766e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:24.576067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f2900-567f-4a1f-aae3-12bd0696766e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.576289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f2900-567f-4a1f-aae3-12bd0696766e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.576611 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f2900-567f-4a1f-aae3-12bd0696766e None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:24.577262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f2900-567f-4a1f-aae3-12bd0696766e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:24.577559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f2900-567f-4a1f-aae3-12bd0696766e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:24.578937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f2900-567f-4a1f-aae3-12bd0696766e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:24.598227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:24.604260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 345/694] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:24.613436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f2900-567f-4a1f-aae3-12bd0696766e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:24.670816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:24.716580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14cb84-6949-4d87-845f-9250fd2502eb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:24.743558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f2900-567f-4a1f-aae3-12bd0696766e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:24.754455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915b2ec-7c07-425b-b0b5-9a49b60e36c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:24.755162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915b2ec-7c07-425b-b0b5-9a49b60e36c3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.755506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915b2ec-7c07-425b-b0b5-9a49b60e36c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.755842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915b2ec-7c07-425b-b0b5-9a49b60e36c3 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:24.756781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915b2ec-7c07-425b-b0b5-9a49b60e36c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:24.757156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915b2ec-7c07-425b-b0b5-9a49b60e36c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:24.758801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915b2ec-7c07-425b-b0b5-9a49b60e36c3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:24.779385 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 350/695] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Aug 19 15:10:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:24.780790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f41dc-a746-4e37-8b16-1d6719e7ecae None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.781170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f41dc-a746-4e37-8b16-1d6719e7ecae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.781540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f41dc-a746-4e37-8b16-1d6719e7ecae None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:24.787149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 351/696] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:24.787149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f41dc-a746-4e37-8b16-1d6719e7ecae None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:24.794020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7d16e-76ab-4e23-98b5-d1aaa636d7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:24.794839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7d16e-76ab-4e23-98b5-d1aaa636d7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.795260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7d16e-76ab-4e23-98b5-d1aaa636d7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.795695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7d16e-76ab-4e23-98b5-d1aaa636d7d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:24.805458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7d16e-76ab-4e23-98b5-d1aaa636d7d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:24.807943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7d16e-76ab-4e23-98b5-d1aaa636d7d7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-16d7d16e-76ab-4e23-98b5-d1aaa636d7d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:24.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:24.825699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:24.826613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:24.826613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:24.826613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:24.826613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:24.826613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:24.826613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:24.826613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:24.826613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:24.830071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 346/697] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:24.831524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a2a08-d33a-464a-b73b-2faeb047d19d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.831987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a2a08-d33a-464a-b73b-2faeb047d19d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.832442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a2a08-d33a-464a-b73b-2faeb047d19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:24.853676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d47a2a08-d33a-464a-b73b-2faeb047d19d None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:24.856514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d47a2a08-d33a-464a-b73b-2faeb047d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:24.904875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:24.906830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f41dc-a746-4e37-8b16-1d6719e7ecae None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:24.948726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 347/698] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:24.948726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a2a08-d33a-464a-b73b-2faeb047d19d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:24.975800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662a2fa-6ac5-455a-97ab-c1b8d9f470cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:24.980363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662a2fa-6ac5-455a-97ab-c1b8d9f470cd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:24.980701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662a2fa-6ac5-455a-97ab-c1b8d9f470cd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:24.980993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662a2fa-6ac5-455a-97ab-c1b8d9f470cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:25.012690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:25.090591 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662a2fa-6ac5-455a-97ab-c1b8d9f470cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:25.092247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662a2fa-6ac5-455a-97ab-c1b8d9f470cd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:25.176939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:25.190555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a2a08-d33a-464a-b73b-2faeb047d19d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:25.209152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 352/699] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 15:10:25.215716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662a2fa-6ac5-455a-97ab-c1b8d9f470cd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:25.249539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2608c-9db3-4948-84e3-bd09c85e174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:25.250528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2608c-9db3-4948-84e3-bd09c85e174a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:25.251033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2608c-9db3-4948-84e3-bd09c85e174a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:25.251523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2608c-9db3-4948-84e3-bd09c85e174a None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:25.252446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2608c-9db3-4948-84e3-bd09c85e174a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:25.252970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2608c-9db3-4948-84e3-bd09c85e174a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:25.255141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2608c-9db3-4948-84e3-bd09c85e174a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:25.270657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 348/700] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:25.278970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2608c-9db3-4948-84e3-bd09c85e174a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:25.423647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662a2fa-6ac5-455a-97ab-c1b8d9f470cd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:25.448480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:25.468748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2608c-9db3-4948-84e3-bd09c85e174a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:25.470570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50026c39-b141-44a0-bc3a-e102e2c3b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:25.471410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50026c39-b141-44a0-bc3a-e102e2c3b72c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:25.471692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50026c39-b141-44a0-bc3a-e102e2c3b72c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:25.471962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50026c39-b141-44a0-bc3a-e102e2c3b72c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:25.484767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50026c39-b141-44a0-bc3a-e102e2c3b72c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:25.487485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50026c39-b141-44a0-bc3a-e102e2c3b72c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-50026c39-b141-44a0-bc3a-e102e2c3b72c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:25.508432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:25.509850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:25.511904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:25.511904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:25.511904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:25.511904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 353/701] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:25.511904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50026c39-b141-44a0-bc3a-e102e2c3b72c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:25.520872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83e38-b7d6-4970-8bab-783855d45ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:25.522269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83e38-b7d6-4970-8bab-783855d45ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:25.522866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83e38-b7d6-4970-8bab-783855d45ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:25.523343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83e38-b7d6-4970-8bab-783855d45ea9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:25.525286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d83e38-b7d6-4970-8bab-783855d45ea9 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=98a63d5b605a4ecfad2e305c491ae755) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:25.526094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d83e38-b7d6-4970-8bab-783855d45ea9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:25.529144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d83e38-b7d6-4970-8bab-783855d45ea9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:25.537883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 349/702] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:25 2022] GET /identity/v3/role_assignments?user.id=98a63d5b605a4ecfad2e305c491ae755&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:25.539553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83e38-b7d6-4970-8bab-783855d45ea9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:25.566531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:25.693962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83e38-b7d6-4970-8bab-783855d45ea9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:25.734447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806729a-107a-40a2-a12b-6cdb4c496294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:25.735465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806729a-107a-40a2-a12b-6cdb4c496294 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:25.735942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806729a-107a-40a2-a12b-6cdb4c496294 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:25.736477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806729a-107a-40a2-a12b-6cdb4c496294 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:25.737345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806729a-107a-40a2-a12b-6cdb4c496294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:25.737850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806729a-107a-40a2-a12b-6cdb4c496294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:25.739455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806729a-107a-40a2-a12b-6cdb4c496294 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:25.745831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50026c39-b141-44a0-bc3a-e102e2c3b72c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:25.755800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 350/703] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:25.764782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806729a-107a-40a2-a12b-6cdb4c496294 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:25.797701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3f347-0132-43be-a395-13f1eb99b4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:25.798354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3f347-0132-43be-a395-13f1eb99b4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:25.798631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3f347-0132-43be-a395-13f1eb99b4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:25.798901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3f347-0132-43be-a395-13f1eb99b4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:25.805425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:25.833220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3f347-0132-43be-a395-13f1eb99b4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:25.834704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3f347-0132-43be-a395-13f1eb99b4b5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-55b3f347-0132-43be-a395-13f1eb99b4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:25.858565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:25.860741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:25.860741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:25.860741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:25.860741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 354/704] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:25.861418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d3f3d-85f2-45a6-8e85-84d1afaa8cb2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:25.861697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d3f3d-85f2-45a6-8e85-84d1afaa8cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:25.861970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d3f3d-85f2-45a6-8e85-84d1afaa8cb2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:25.864080 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 355/705] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:25.870617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e230b-3a54-4ef8-bf83-0325dd7954a2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:25.871109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e230b-3a54-4ef8-bf83-0325dd7954a2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:25.871520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e230b-3a54-4ef8-bf83-0325dd7954a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:25.895520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-083e230b-3a54-4ef8-bf83-0325dd7954a2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:25.896985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-083e230b-3a54-4ef8-bf83-0325dd7954a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:25.948751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806729a-107a-40a2-a12b-6cdb4c496294 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:25.983148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 356/706] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:25.984240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e230b-3a54-4ef8-bf83-0325dd7954a2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:25.998441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:26.008468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48e14d-2651-4d61-82b3-9863d665fd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:26.009499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48e14d-2651-4d61-82b3-9863d665fd66 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:26.010731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48e14d-2651-4d61-82b3-9863d665fd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:26.011189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48e14d-2651-4d61-82b3-9863d665fd66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:26.031436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48e14d-2651-4d61-82b3-9863d665fd66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:26.032967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48e14d-2651-4d61-82b3-9863d665fd66 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9e48e14d-2651-4d61-82b3-9863d665fd66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:26.060663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:26.061716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:26.062803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:26.062803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:26.062803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:26.062803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:26.062803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:26.062803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:26.062803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:26.062803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:26.062803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 351/707] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:26.068190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48e14d-2651-4d61-82b3-9863d665fd66 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:26.099072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e230b-3a54-4ef8-bf83-0325dd7954a2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:26.134666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc932b-a5ab-4af8-aaa0-b32862ca456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:26.136257 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc932b-a5ab-4af8-aaa0-b32862ca456e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:26.136696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc932b-a5ab-4af8-aaa0-b32862ca456e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:26.137107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc932b-a5ab-4af8-aaa0-b32862ca456e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:26.138078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc932b-a5ab-4af8-aaa0-b32862ca456e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:26.138531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc932b-a5ab-4af8-aaa0-b32862ca456e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:26.140051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc932b-a5ab-4af8-aaa0-b32862ca456e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:26.152982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 357/708] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:26.158898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc932b-a5ab-4af8-aaa0-b32862ca456e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:26.208480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:26.243603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48e14d-2651-4d61-82b3-9863d665fd66 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:26.287912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97e47-686f-42d5-b83c-ce027d8832e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:26.289458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97e47-686f-42d5-b83c-ce027d8832e8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:26.289970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97e47-686f-42d5-b83c-ce027d8832e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:26.290381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a97e47-686f-42d5-b83c-ce027d8832e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:26.291559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a97e47-686f-42d5-b83c-ce027d8832e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:26.292075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a97e47-686f-42d5-b83c-ce027d8832e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:26.294102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a97e47-686f-42d5-b83c-ce027d8832e8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:26.309835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 352/709] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:26.320753 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97e47-686f-42d5-b83c-ce027d8832e8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:26.321499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:26.378991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc932b-a5ab-4af8-aaa0-b32862ca456e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:26.417045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d47ed-5106-41af-83f3-f88bc871e11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:26.417707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d47ed-5106-41af-83f3-f88bc871e11c None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:26.417984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d47ed-5106-41af-83f3-f88bc871e11c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:26.418258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d47ed-5106-41af-83f3-f88bc871e11c None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:26.419013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d47ed-5106-41af-83f3-f88bc871e11c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:26.420593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d47ed-5106-41af-83f3-f88bc871e11c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:26.441308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a97e47-686f-42d5-b83c-ce027d8832e8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:26.444449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 358/710] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:10:26 2022] POST /identity/v3/users => generated 254 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:26.444449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821a76b-b5e0-44ea-a84e-a6933c6075b9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:26.444677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821a76b-b5e0-44ea-a84e-a6933c6075b9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:26.445837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821a76b-b5e0-44ea-a84e-a6933c6075b9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:26.450677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 359/711] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:26.450677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821a76b-b5e0-44ea-a84e-a6933c6075b9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:26.484795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:26.496330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d09fe-07d0-4420-9cfe-104ad7904cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:26.497789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d09fe-07d0-4420-9cfe-104ad7904cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:26.498346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d09fe-07d0-4420-9cfe-104ad7904cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:26.499170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d09fe-07d0-4420-9cfe-104ad7904cf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:26.500796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d09fe-07d0-4420-9cfe-104ad7904cf9 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=383ff491387642a3ab45ab692f94189c) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:26.501655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d09fe-07d0-4420-9cfe-104ad7904cf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:26.504236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d09fe-07d0-4420-9cfe-104ad7904cf9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:26.513817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 353/712] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:26 2022] GET /identity/v3/role_assignments?user.id=383ff491387642a3ab45ab692f94189c&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:26.515576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbbd11-681d-4b7b-9953-90a14262162e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:26.516946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbbd11-681d-4b7b-9953-90a14262162e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:26.517457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbbd11-681d-4b7b-9953-90a14262162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:26.539745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fdfbbd11-681d-4b7b-9953-90a14262162e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:26.542424 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fdfbbd11-681d-4b7b-9953-90a14262162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:26.596629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821a76b-b5e0-44ea-a84e-a6933c6075b9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:26.628198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 354/713] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:26.633284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:26.643262 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93034fae-8561-4e1d-8cf3-5afe392f3cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:26.643907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93034fae-8561-4e1d-8cf3-5afe392f3cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:26.644179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93034fae-8561-4e1d-8cf3-5afe392f3cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:26.644555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93034fae-8561-4e1d-8cf3-5afe392f3cb5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:26.645148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93034fae-8561-4e1d-8cf3-5afe392f3cb5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:26.646595 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93034fae-8561-4e1d-8cf3-5afe392f3cb5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-93034fae-8561-4e1d-8cf3-5afe392f3cb5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:26.659553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:26.660794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:26.662061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 360/714] 10.222.0.75 () {56 vars in 1129 bytes} [Fri Aug 19 15:10:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:26.667525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbbd11-681d-4b7b-9953-90a14262162e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:26.779081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbbd11-681d-4b7b-9953-90a14262162e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:26.806729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:26.818144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eec6a-9b95-4e07-a7fb-69b7dac799cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:26.818754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eec6a-9b95-4e07-a7fb-69b7dac799cb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:26.819089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eec6a-9b95-4e07-a7fb-69b7dac799cb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:26.819358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eec6a-9b95-4e07-a7fb-69b7dac799cb None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:26.820035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eec6a-9b95-4e07-a7fb-69b7dac799cb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:26.820339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eec6a-9b95-4e07-a7fb-69b7dac799cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:26.821740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eec6a-9b95-4e07-a7fb-69b7dac799cb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:26.827081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 355/715] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Aug 19 15:10:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:26.832572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93034fae-8561-4e1d-8cf3-5afe392f3cb5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:26.921689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eec6a-9b95-4e07-a7fb-69b7dac799cb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:26.957104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93034fae-8561-4e1d-8cf3-5afe392f3cb5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:27.012679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26af50f-faf6-4755-8f5e-13d111323f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:27.013302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26af50f-faf6-4755-8f5e-13d111323f5a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:27.013584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26af50f-faf6-4755-8f5e-13d111323f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:27.013862 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26af50f-faf6-4755-8f5e-13d111323f5a None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:27.014947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26af50f-faf6-4755-8f5e-13d111323f5a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:27.015255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26af50f-faf6-4755-8f5e-13d111323f5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:27.016984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26af50f-faf6-4755-8f5e-13d111323f5a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:27.025491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 361/716] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Aug 19 15:10:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:27.061349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eec6a-9b95-4e07-a7fb-69b7dac799cb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:27.102138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddda44-d42d-4c1f-92d7-0945696d42fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:27.103141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddda44-d42d-4c1f-92d7-0945696d42fd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:27.103643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddda44-d42d-4c1f-92d7-0945696d42fd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:27.104433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddda44-d42d-4c1f-92d7-0945696d42fd None admin] PATH_INFO: `/v3/services/network` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:27.105488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddda44-d42d-4c1f-92d7-0945696d42fd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:27.107339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddda44-d42d-4c1f-92d7-0945696d42fd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-38ddda44-d42d-4c1f-92d7-0945696d42fd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:27.120669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:27.121840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 15:10:27.122687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:27.128942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 356/717] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:26 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:27.132069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26af50f-faf6-4755-8f5e-13d111323f5a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:27.199049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:27.254350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26af50f-faf6-4755-8f5e-13d111323f5a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:27.292791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a3111-5ee6-41be-aa44-c0dd34c7b545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:27.293406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a3111-5ee6-41be-aa44-c0dd34c7b545 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:27.293689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a3111-5ee6-41be-aa44-c0dd34c7b545 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:27.293964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a3111-5ee6-41be-aa44-c0dd34c7b545 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:27.294606 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a3111-5ee6-41be-aa44-c0dd34c7b545 None admin] RBAC: Adding query filter params (name=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:27.294912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a3111-5ee6-41be-aa44-c0dd34c7b545 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:27.296375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a3111-5ee6-41be-aa44-c0dd34c7b545 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:27.301491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 362/718] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:27.307604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddda44-d42d-4c1f-92d7-0945696d42fd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:27.388310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:27.438374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddda44-d42d-4c1f-92d7-0945696d42fd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:27.480608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb601b4-4118-4b66-a3a8-966b0c593dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:27.481228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb601b4-4118-4b66-a3a8-966b0c593dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:27.481512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb601b4-4118-4b66-a3a8-966b0c593dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:27.481788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb601b4-4118-4b66-a3a8-966b0c593dc1 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:27.482431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb601b4-4118-4b66-a3a8-966b0c593dc1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:27.482736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb601b4-4118-4b66-a3a8-966b0c593dc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:27.484131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb601b4-4118-4b66-a3a8-966b0c593dc1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:27.490922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 357/719] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:28.038720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012887-a909-445a-aea3-a2058f4a6109 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:28.039238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012887-a909-445a-aea3-a2058f4a6109 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:28.039315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012887-a909-445a-aea3-a2058f4a6109 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:28.041071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 363/720] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:28.046806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1265c-2bc7-469f-8723-02ae526cb717 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:28.047348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1265c-2bc7-469f-8723-02ae526cb717 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:28.047428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1265c-2bc7-469f-8723-02ae526cb717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:28.066687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6ba1265c-2bc7-469f-8723-02ae526cb717 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:28.067882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6ba1265c-2bc7-469f-8723-02ae526cb717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:28.160742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 358/721] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:28.511493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012887-a909-445a-aea3-a2058f4a6109 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:28.641844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012887-a909-445a-aea3-a2058f4a6109 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:28.689105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:28.704805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bf39f-fc73-4f23-bca0-6e8a3c22aa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:28.705742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bf39f-fc73-4f23-bca0-6e8a3c22aa29 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:28.706216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bf39f-fc73-4f23-bca0-6e8a3c22aa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:28.706680 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bf39f-fc73-4f23-bca0-6e8a3c22aa29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:28.722885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bf39f-fc73-4f23-bca0-6e8a3c22aa29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-738bf39f-fc73-4f23-bca0-6e8a3c22aa29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:28.727833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:28.729388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:28.730560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:28.732885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 364/722] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:28.738950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1265c-2bc7-469f-8723-02ae526cb717 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:28.818232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:28.890297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1265c-2bc7-469f-8723-02ae526cb717 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:28.936019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b0370-afee-4d43-ae99-41c3ba511a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:28.937029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b0370-afee-4d43-ae99-41c3ba511a43 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:28.937508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b0370-afee-4d43-ae99-41c3ba511a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:28.937975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b0370-afee-4d43-ae99-41c3ba511a43 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:28.938861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b0370-afee-4d43-ae99-41c3ba511a43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:28.939376 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b0370-afee-4d43-ae99-41c3ba511a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:28.940990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b0370-afee-4d43-ae99-41c3ba511a43 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:28.951183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 359/723] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:28.956425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bf39f-fc73-4f23-bca0-6e8a3c22aa29 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:29.086260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bf39f-fc73-4f23-bca0-6e8a3c22aa29 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:29.132256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583db98-8266-40ae-b67c-443d00ebc500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:29.132904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583db98-8266-40ae-b67c-443d00ebc500 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.133178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583db98-8266-40ae-b67c-443d00ebc500 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.133446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583db98-8266-40ae-b67c-443d00ebc500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.152867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583db98-8266-40ae-b67c-443d00ebc500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:29.154343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583db98-8266-40ae-b67c-443d00ebc500 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4583db98-8266-40ae-b67c-443d00ebc500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:29.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:29.174936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:29.175833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:29.175833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:29.175833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:29.175833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 365/724] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:29.180864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b0370-afee-4d43-ae99-41c3ba511a43 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:29.318018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0ab12-68be-4cc9-9cd4-9b59d84ded36 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.318646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0ab12-68be-4cc9-9cd4-9b59d84ded36 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.319113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0ab12-68be-4cc9-9cd4-9b59d84ded36 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.326530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 366/725] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:29.331686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3599-8788-4175-a5fe-9492876a1a8b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.332262 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3599-8788-4175-a5fe-9492876a1a8b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.332760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3599-8788-4175-a5fe-9492876a1a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.363448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b0370-afee-4d43-ae99-41c3ba511a43 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:29.375101 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:29.380294 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5c5f3599-8788-4175-a5fe-9492876a1a8b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:29.381510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c5f3599-8788-4175-a5fe-9492876a1a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:29.407322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f311162-5f40-45e4-8ec9-461c3de20971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:29.408440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311162-5f40-45e4-8ec9-461c3de20971 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.408872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311162-5f40-45e4-8ec9-461c3de20971 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.409538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311162-5f40-45e4-8ec9-461c3de20971 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.410650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:29.412191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f311162-5f40-45e4-8ec9-461c3de20971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:29.412676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f311162-5f40-45e4-8ec9-461c3de20971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:29.414802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f311162-5f40-45e4-8ec9-461c3de20971 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:29.429713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 360/726] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:29.431282 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7142c35-d399-4d37-8fdb-1a96b427cde7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.431768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7142c35-d399-4d37-8fdb-1a96b427cde7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.432201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7142c35-d399-4d37-8fdb-1a96b427cde7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.438211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 361/727] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:29.439340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7142c35-d399-4d37-8fdb-1a96b427cde7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:29.471203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 367/728] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:29.471203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80fe2c-f016-4388-b5f6-5ec2f4054dfe None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.471203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80fe2c-f016-4388-b5f6-5ec2f4054dfe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.471203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80fe2c-f016-4388-b5f6-5ec2f4054dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.488822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5d80fe2c-f016-4388-b5f6-5ec2f4054dfe None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:29.492438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5d80fe2c-f016-4388-b5f6-5ec2f4054dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:29.577859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 368/729] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:29.579222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f9f16-4363-4711-a8df-83f4c805d62e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.579503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f9f16-4363-4711-a8df-83f4c805d62e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.579773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f9f16-4363-4711-a8df-83f4c805d62e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.585579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 369/730] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:29.586675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a28b9f-f8b8-4bdc-84c2-90877796b1f7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.586951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a28b9f-f8b8-4bdc-84c2-90877796b1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.587310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a28b9f-f8b8-4bdc-84c2-90877796b1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.602062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7142c35-d399-4d37-8fdb-1a96b427cde7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:29.623775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-66a28b9f-f8b8-4bdc-84c2-90877796b1f7 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:29.624922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66a28b9f-f8b8-4bdc-84c2-90877796b1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:29.679356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18315911-efb8-4c89-9d85-0a59c14ad0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:29.680002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18315911-efb8-4c89-9d85-0a59c14ad0ae None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.680276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18315911-efb8-4c89-9d85-0a59c14ad0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.680569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18315911-efb8-4c89-9d85-0a59c14ad0ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.695888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18315911-efb8-4c89-9d85-0a59c14ad0ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:29.697947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18315911-efb8-4c89-9d85-0a59c14ad0ae None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-18315911-efb8-4c89-9d85-0a59c14ad0ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:29.726141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:29.727151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:29.728117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 362/731] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Aug 19 15:10:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:29.733411 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18315911-efb8-4c89-9d85-0a59c14ad0ae None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:29.752083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 370/732] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 15:10:29.793549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881851c5-e240-4dc3-a6b8-4427012e5b56 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.793851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881851c5-e240-4dc3-a6b8-4427012e5b56 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.794143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881851c5-e240-4dc3-a6b8-4427012e5b56 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.795469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 371/733] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:10:29.801597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a79cf4-a640-453f-af10-54030903fc5f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.801881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a79cf4-a640-453f-af10-54030903fc5f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.802155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a79cf4-a640-453f-af10-54030903fc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.823343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-00a79cf4-a640-453f-af10-54030903fc5f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:29.824580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-00a79cf4-a640-453f-af10-54030903fc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:29.840774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:29.846757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:29.900644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18315911-efb8-4c89-9d85-0a59c14ad0ae None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:29.909778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 372/734] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 15:10:29.910941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a79cf4-a640-453f-af10-54030903fc5f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:29.953118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fb6d3-24e1-43aa-9b89-2b317cc61ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:29.954006 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fb6d3-24e1-43aa-9b89-2b317cc61ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:29.954299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fb6d3-24e1-43aa-9b89-2b317cc61ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:29.954631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fb6d3-24e1-43aa-9b89-2b317cc61ef9 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:29.955797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fb6d3-24e1-43aa-9b89-2b317cc61ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:29.956111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fb6d3-24e1-43aa-9b89-2b317cc61ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:29.957856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6fb6d3-24e1-43aa-9b89-2b317cc61ef9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:29.984966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 363/735] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Aug 19 15:10:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:29.985879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fb6d3-24e1-43aa-9b89-2b317cc61ef9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.039465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a79cf4-a640-453f-af10-54030903fc5f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.078288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0f774-ec57-449e-a011-61c2c14d3ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.078907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0f774-ec57-449e-a011-61c2c14d3ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.079181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0f774-ec57-449e-a011-61c2c14d3ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.079446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0f774-ec57-449e-a011-61c2c14d3ac2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.090218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0f774-ec57-449e-a011-61c2c14d3ac2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0ad0f774-ec57-449e-a011-61c2c14d3ac2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:30.094821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:30.095757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:30.096648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:30.097287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:30.097361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 373/736] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:30.097361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0f774-ec57-449e-a011-61c2c14d3ac2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.112289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6fb6d3-24e1-43aa-9b89-2b317cc61ef9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.113988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:30.148978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e71d9-e67d-4a42-8fb8-09edf25c5075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.149606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e71d9-e67d-4a42-8fb8-09edf25c5075 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.149888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e71d9-e67d-4a42-8fb8-09edf25c5075 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.150161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e71d9-e67d-4a42-8fb8-09edf25c5075 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.162546 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e71d9-e67d-4a42-8fb8-09edf25c5075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b73e71d9-e67d-4a42-8fb8-09edf25c5075 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:30.167571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:30.168934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:30.170423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:30.173529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 364/737] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:30.174340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e71d9-e67d-4a42-8fb8-09edf25c5075 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.212623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0f774-ec57-449e-a011-61c2c14d3ac2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.251722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b45ea-296a-439f-b067-e5d04b6913b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.252352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b45ea-296a-439f-b067-e5d04b6913b3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.252666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b45ea-296a-439f-b067-e5d04b6913b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.252941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b45ea-296a-439f-b067-e5d04b6913b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.263783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b45ea-296a-439f-b067-e5d04b6913b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:30.265269 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b45ea-296a-439f-b067-e5d04b6913b3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:30.274542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e71d9-e67d-4a42-8fb8-09edf25c5075 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-537b45ea-296a-439f-b067-e5d04b6913b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:30.280420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:30.281563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:30.282482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:30.282482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:30.282482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:30.284660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 374/738] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:30.285459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b45ea-296a-439f-b067-e5d04b6913b3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.310646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95e51-4ec0-4f56-8cd1-569865b11dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.311245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95e51-4ec0-4f56-8cd1-569865b11dfd None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.311522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95e51-4ec0-4f56-8cd1-569865b11dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.311798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95e51-4ec0-4f56-8cd1-569865b11dfd None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.312478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95e51-4ec0-4f56-8cd1-569865b11dfd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:30.313851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95e51-4ec0-4f56-8cd1-569865b11dfd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:30.332514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 365/739] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:10:30 2022] POST /identity/v3/services => generated 251 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:30.332514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95e51-4ec0-4f56-8cd1-569865b11dfd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.344185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:30.354110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:30.402761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b45ea-296a-439f-b067-e5d04b6913b3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.448826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e851ba5-b7f1-4db7-a5d0-da4a09b91654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.449366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e851ba5-b7f1-4db7-a5d0-da4a09b91654 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.449647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e851ba5-b7f1-4db7-a5d0-da4a09b91654 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.449912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e851ba5-b7f1-4db7-a5d0-da4a09b91654 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.450587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e851ba5-b7f1-4db7-a5d0-da4a09b91654 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:30.450883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e851ba5-b7f1-4db7-a5d0-da4a09b91654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:30.452364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e851ba5-b7f1-4db7-a5d0-da4a09b91654 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:30.456999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95e51-4ec0-4f56-8cd1-569865b11dfd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.466593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 375/740] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:30.466593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e851ba5-b7f1-4db7-a5d0-da4a09b91654 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.503985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3206ad-37c9-472c-9649-b9451b77313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.504631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3206ad-37c9-472c-9649-b9451b77313d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.505021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3206ad-37c9-472c-9649-b9451b77313d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.505413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3206ad-37c9-472c-9649-b9451b77313d None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.506088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3206ad-37c9-472c-9649-b9451b77313d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:30.506390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3206ad-37c9-472c-9649-b9451b77313d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:30.507773 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3206ad-37c9-472c-9649-b9451b77313d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:30.518348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 366/741] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:30.519290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3206ad-37c9-472c-9649-b9451b77313d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.582787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e851ba5-b7f1-4db7-a5d0-da4a09b91654 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.593607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:30.605148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:30.622336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9875e2-d560-4589-a774-b9b0b733a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.622945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9875e2-d560-4589-a774-b9b0b733a5e9 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.623223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9875e2-d560-4589-a774-b9b0b733a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.627481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9875e2-d560-4589-a774-b9b0b733a5e9 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.627481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9875e2-d560-4589-a774-b9b0b733a5e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:30.627481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9875e2-d560-4589-a774-b9b0b733a5e9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:30.640534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3206ad-37c9-472c-9649-b9451b77313d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.643254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 376/742] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:10:30 2022] POST /identity/v3/services => generated 240 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:30.644183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9875e2-d560-4589-a774-b9b0b733a5e9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.713496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36837d1-20b1-46a9-a361-839981c1d86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.714622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36837d1-20b1-46a9-a361-839981c1d86f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.715109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36837d1-20b1-46a9-a361-839981c1d86f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.715685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36837d1-20b1-46a9-a361-839981c1d86f None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.716607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36837d1-20b1-46a9-a361-839981c1d86f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:30.717216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36837d1-20b1-46a9-a361-839981c1d86f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:30.718905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36837d1-20b1-46a9-a361-839981c1d86f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:30.733028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 367/743] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:30.734362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36837d1-20b1-46a9-a361-839981c1d86f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.794015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9875e2-d560-4589-a774-b9b0b733a5e9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.845623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960dca8-63ff-4621-b49c-7d2aa690fe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.846253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960dca8-63ff-4621-b49c-7d2aa690fe2a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.846534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960dca8-63ff-4621-b49c-7d2aa690fe2a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.846807 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960dca8-63ff-4621-b49c-7d2aa690fe2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.855827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36837d1-20b1-46a9-a361-839981c1d86f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:30.862000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960dca8-63ff-4621-b49c-7d2aa690fe2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:30.863476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960dca8-63ff-4621-b49c-7d2aa690fe2a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6960dca8-63ff-4621-b49c-7d2aa690fe2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:30.882906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:30.883923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:30.884892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:30.884892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:30.884892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:30.884892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 377/744] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:30.886382 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960dca8-63ff-4621-b49c-7d2aa690fe2a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:30.887057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:30.899259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c56212-3f04-4107-a1c6-e1dc7fcf6cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:30.899891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c56212-3f04-4107-a1c6-e1dc7fcf6cac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:30.900307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c56212-3f04-4107-a1c6-e1dc7fcf6cac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:30.900748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c56212-3f04-4107-a1c6-e1dc7fcf6cac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:30.913664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c56212-3f04-4107-a1c6-e1dc7fcf6cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:30.915385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c56212-3f04-4107-a1c6-e1dc7fcf6cac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:30.921956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-62c56212-3f04-4107-a1c6-e1dc7fcf6cac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:30.933983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:30.934995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:30.935907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:30.935907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:30.935907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:30.936259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 368/745] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:30.941696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c56212-3f04-4107-a1c6-e1dc7fcf6cac None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.007050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960dca8-63ff-4621-b49c-7d2aa690fe2a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:31.046004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8f0bb-0c10-4b93-b7a3-e5cfec45ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:31.046647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8f0bb-0c10-4b93-b7a3-e5cfec45ce9f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:31.046930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8f0bb-0c10-4b93-b7a3-e5cfec45ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:31.047207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8f0bb-0c10-4b93-b7a3-e5cfec45ce9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:31.054319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8f0bb-0c10-4b93-b7a3-e5cfec45ce9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:31.055780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8f0bb-0c10-4b93-b7a3-e5cfec45ce9f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e9c8f0bb-0c10-4b93-b7a3-e5cfec45ce9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:31.068184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:31.069223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:31.070128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:31.070128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:31.070128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:31.070128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:31.070128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:31.070128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:31.070128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:31.070128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:31.074218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 378/746] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:31.074345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c56212-3f04-4107-a1c6-e1dc7fcf6cac None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:31.075090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8f0bb-0c10-4b93-b7a3-e5cfec45ce9f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.118183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85298f1-0748-4396-9c2a-65ca5d238092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:31.119857 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85298f1-0748-4396-9c2a-65ca5d238092 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:31.120259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85298f1-0748-4396-9c2a-65ca5d238092 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:31.120647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85298f1-0748-4396-9c2a-65ca5d238092 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:31.121335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85298f1-0748-4396-9c2a-65ca5d238092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:31.122096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85298f1-0748-4396-9c2a-65ca5d238092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:31.123926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85298f1-0748-4396-9c2a-65ca5d238092 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:31.140702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 369/747] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:31.142236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85298f1-0748-4396-9c2a-65ca5d238092 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.183197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:31.194136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8f0bb-0c10-4b93-b7a3-e5cfec45ce9f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:31.197114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:31.237111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ee745-8ca9-46de-bc9b-5ce5ba71855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:31.237723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ee745-8ca9-46de-bc9b-5ce5ba71855b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:31.238002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ee745-8ca9-46de-bc9b-5ce5ba71855b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:31.238278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ee745-8ca9-46de-bc9b-5ce5ba71855b None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:31.238925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ee745-8ca9-46de-bc9b-5ce5ba71855b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:31.239227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ee745-8ca9-46de-bc9b-5ce5ba71855b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:31.240632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ee745-8ca9-46de-bc9b-5ce5ba71855b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:31.258870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 379/748] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:31.258870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ee745-8ca9-46de-bc9b-5ce5ba71855b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.282802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85298f1-0748-4396-9c2a-65ca5d238092 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:31.329088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361810f-5b19-453f-ba75-b1600dfcee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:31.329655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361810f-5b19-453f-ba75-b1600dfcee0d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:31.330005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361810f-5b19-453f-ba75-b1600dfcee0d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:31.330351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361810f-5b19-453f-ba75-b1600dfcee0d None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:31.331114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361810f-5b19-453f-ba75-b1600dfcee0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:31.331487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361810f-5b19-453f-ba75-b1600dfcee0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:31.333045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361810f-5b19-453f-ba75-b1600dfcee0d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:31.345435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 370/749] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:31.346704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361810f-5b19-453f-ba75-b1600dfcee0d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.379992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ee745-8ca9-46de-bc9b-5ce5ba71855b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:31.427572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1e8b0-33c7-4ed5-b23e-7f52d3f0d1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:31.428692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1e8b0-33c7-4ed5-b23e-7f52d3f0d1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:31.429175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1e8b0-33c7-4ed5-b23e-7f52d3f0d1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:31.429686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1e8b0-33c7-4ed5-b23e-7f52d3f0d1d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:31.437523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1e8b0-33c7-4ed5-b23e-7f52d3f0d1d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:31.439324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1e8b0-33c7-4ed5-b23e-7f52d3f0d1d1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-15e1e8b0-33c7-4ed5-b23e-7f52d3f0d1d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:31.451279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:31.452217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:31.453198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:31.459905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:31.460478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361810f-5b19-453f-ba75-b1600dfcee0d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:31.460689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 380/750] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:10:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:31.461706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1e8b0-33c7-4ed5-b23e-7f52d3f0d1d1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.480290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:31.507943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826c455-e379-4b1b-960d-a751fad27b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:31.508771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826c455-e379-4b1b-960d-a751fad27b24 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:31.509209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826c455-e379-4b1b-960d-a751fad27b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:31.509548 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826c455-e379-4b1b-960d-a751fad27b24 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:31.520047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826c455-e379-4b1b-960d-a751fad27b24 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:31.522196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826c455-e379-4b1b-960d-a751fad27b24 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a826c455-e379-4b1b-960d-a751fad27b24 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:31.536589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:31.537724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:31.538795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:31.547634 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 371/751] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:31.548710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826c455-e379-4b1b-960d-a751fad27b24 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.591479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1e8b0-33c7-4ed5-b23e-7f52d3f0d1d1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:31.635322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eeb845-cb7a-4781-9623-e16431017a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:31.635990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeb845-cb7a-4781-9623-e16431017a15 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:31.636268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeb845-cb7a-4781-9623-e16431017a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:31.636571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eeb845-cb7a-4781-9623-e16431017a15 None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/98a63d5b605a4ecfad2e305c491ae755/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:31.639175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eeb845-cb7a-4781-9623-e16431017a15 None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=98a63d5b605a4ecfad2e305c491ae755, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:31.640933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eeb845-cb7a-4781-9623-e16431017a15 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:31.662410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826c455-e379-4b1b-960d-a751fad27b24 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:31.670548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 381/752] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:10:31 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/98a63d5b605a4ecfad2e305c491ae755/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:10:31.670548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eeb845-cb7a-4781-9623-e16431017a15 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.742882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd1300-1169-48ae-b90f-f10705310eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:31.743510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd1300-1169-48ae-b90f-f10705310eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:31.743790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd1300-1169-48ae-b90f-f10705310eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:31.744060 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd1300-1169-48ae-b90f-f10705310eb7 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:31.744885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd1300-1169-48ae-b90f-f10705310eb7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:31.745193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd1300-1169-48ae-b90f-f10705310eb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:31.752170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd1300-1169-48ae-b90f-f10705310eb7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:31.787146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 372/753] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Aug 19 15:10:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:31.793388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd1300-1169-48ae-b90f-f10705310eb7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.803134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:31.844902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eeb845-cb7a-4781-9623-e16431017a15 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:31.848518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:31.903211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e035356-e5d7-48fa-b4bb-f3298311b0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:31.903876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e035356-e5d7-48fa-b4bb-f3298311b0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:31.904159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e035356-e5d7-48fa-b4bb-f3298311b0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:31.904528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e035356-e5d7-48fa-b4bb-f3298311b0e5 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:31.905271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e035356-e5d7-48fa-b4bb-f3298311b0e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:31.905577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e035356-e5d7-48fa-b4bb-f3298311b0e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:31.907812 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e035356-e5d7-48fa-b4bb-f3298311b0e5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:31.929813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 382/754] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Aug 19 15:10:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:31.935469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e035356-e5d7-48fa-b4bb-f3298311b0e5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:31.968528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd1300-1169-48ae-b90f-f10705310eb7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:32.020442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a227009-11a4-4f20-9ab7-0183aefc8a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:32.021743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a227009-11a4-4f20-9ab7-0183aefc8a01 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.023467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a227009-11a4-4f20-9ab7-0183aefc8a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.023959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a227009-11a4-4f20-9ab7-0183aefc8a01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.110222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a227009-11a4-4f20-9ab7-0183aefc8a01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:32.111266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e035356-e5d7-48fa-b4bb-f3298311b0e5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:32.112313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a227009-11a4-4f20-9ab7-0183aefc8a01 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:32.132805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:32.194508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:32.221130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a417596-68b5-4ae8-b8be-88f264fd7326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:32.221802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a417596-68b5-4ae8-b8be-88f264fd7326 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.222030 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a417596-68b5-4ae8-b8be-88f264fd7326 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.222304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a417596-68b5-4ae8-b8be-88f264fd7326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.231211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 373/755] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 15:10:32.237892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a227009-11a4-4f20-9ab7-0183aefc8a01 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:32.349224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a417596-68b5-4ae8-b8be-88f264fd7326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:32.350707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a417596-68b5-4ae8-b8be-88f264fd7326 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:32.417762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a227009-11a4-4f20-9ab7-0183aefc8a01 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:32.435867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:32.477528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 383/756] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 15:10:32.478952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59cdd5-189f-43a1-8004-5e471ee86a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:32.479607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59cdd5-189f-43a1-8004-5e471ee86a2c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.479888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59cdd5-189f-43a1-8004-5e471ee86a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.480164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59cdd5-189f-43a1-8004-5e471ee86a2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.483058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a417596-68b5-4ae8-b8be-88f264fd7326 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:32.497784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59cdd5-189f-43a1-8004-5e471ee86a2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:32.499260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59cdd5-189f-43a1-8004-5e471ee86a2c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3a59cdd5-189f-43a1-8004-5e471ee86a2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:32.515209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:32.516768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:32.517693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:32.517693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:32.517693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:32.517693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 374/757] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:32.518336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c32150-5853-43ad-ab96-d284446ef7cf None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.518503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c32150-5853-43ad-ab96-d284446ef7cf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.520423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c32150-5853-43ad-ab96-d284446ef7cf None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.520423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 375/758] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:32.522805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c32150-5853-43ad-ab96-d284446ef7cf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:32.525914 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:32.625252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a417596-68b5-4ae8-b8be-88f264fd7326 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:32.636532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c32150-5853-43ad-ab96-d284446ef7cf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:32.680001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361806a-9f76-42e6-9c4f-c1d3b8e4498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:32.680684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361806a-9f76-42e6-9c4f-c1d3b8e4498d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.680959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361806a-9f76-42e6-9c4f-c1d3b8e4498d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.681232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361806a-9f76-42e6-9c4f-c1d3b8e4498d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.682645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a679c-70ff-484b-89a1-6dc228fff542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:32.683251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a679c-70ff-484b-89a1-6dc228fff542 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.683525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a679c-70ff-484b-89a1-6dc228fff542 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.683790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a679c-70ff-484b-89a1-6dc228fff542 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.684465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a679c-70ff-484b-89a1-6dc228fff542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:32.684837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a679c-70ff-484b-89a1-6dc228fff542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:32.686221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a679c-70ff-484b-89a1-6dc228fff542 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:32.698406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361806a-9f76-42e6-9c4f-c1d3b8e4498d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:32.704437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361806a-9f76-42e6-9c4f-c1d3b8e4498d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:32.704591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 376/759] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:32.704591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e7378-c4cc-4279-b2ea-4b19297290f9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.704591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e7378-c4cc-4279-b2ea-4b19297290f9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.704591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e7378-c4cc-4279-b2ea-4b19297290f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.723087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b76e7378-c4cc-4279-b2ea-4b19297290f9 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:32.724318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b76e7378-c4cc-4279-b2ea-4b19297290f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d361806a-9f76-42e6-9c4f-c1d3b8e4498d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:32.724615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:32.725528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:32.726423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:32.726423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:32.726423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:32.734480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 384/760] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:32.735652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361806a-9f76-42e6-9c4f-c1d3b8e4498d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:32.751197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:32.802328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 377/761] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:32.803565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e7378-c4cc-4279-b2ea-4b19297290f9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:32.825965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:32.846530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361806a-9f76-42e6-9c4f-c1d3b8e4498d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:32.882780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c278ca9f-0208-4bde-9782-3ee8c26d32f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:32.884166 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278ca9f-0208-4bde-9782-3ee8c26d32f1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.884879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278ca9f-0208-4bde-9782-3ee8c26d32f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.885486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278ca9f-0208-4bde-9782-3ee8c26d32f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.893225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278ca9f-0208-4bde-9782-3ee8c26d32f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:32.894883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c278ca9f-0208-4bde-9782-3ee8c26d32f1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c278ca9f-0208-4bde-9782-3ee8c26d32f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:32.907534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:32.908670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:32.909784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:32.909784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:32.909784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:32.909784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:32.909784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:32.909784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:32.909784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:32.909784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:32.916864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 385/762] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:32.918333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49255a32-59c2-4a3f-96a6-39aba060fdca None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.918793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49255a32-59c2-4a3f-96a6-39aba060fdca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.919071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49255a32-59c2-4a3f-96a6-39aba060fdca None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.920757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 386/763] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:32.921469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49255a32-59c2-4a3f-96a6-39aba060fdca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:32.934128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e7378-c4cc-4279-b2ea-4b19297290f9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:32.973494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1e18b-b04e-4d5a-bfdc-ae664f15ee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:32.974106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1e18b-b04e-4d5a-bfdc-ae664f15ee8f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.974386 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1e18b-b04e-4d5a-bfdc-ae664f15ee8f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.974667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1e18b-b04e-4d5a-bfdc-ae664f15ee8f None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:32.975539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1e18b-b04e-4d5a-bfdc-ae664f15ee8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:32.975889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1e18b-b04e-4d5a-bfdc-ae664f15ee8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:32.977832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1e18b-b04e-4d5a-bfdc-ae664f15ee8f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:32.996949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 378/764] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:32.998498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944613d-5981-4767-b19c-0f17679049c2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:32.998792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944613d-5981-4767-b19c-0f17679049c2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:32.999118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944613d-5981-4767-b19c-0f17679049c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.021782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1944613d-5981-4767-b19c-0f17679049c2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:33.023097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1944613d-5981-4767-b19c-0f17679049c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:33.039094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:33.048637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49255a32-59c2-4a3f-96a6-39aba060fdca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:33.096113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc931c1b-d8c0-4d83-92cd-d49436fe7044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:33.097036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931c1b-d8c0-4d83-92cd-d49436fe7044 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:33.097497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931c1b-d8c0-4d83-92cd-d49436fe7044 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:33.097915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc931c1b-d8c0-4d83-92cd-d49436fe7044 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.098735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc931c1b-d8c0-4d83-92cd-d49436fe7044 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:33.099183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc931c1b-d8c0-4d83-92cd-d49436fe7044 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:33.099792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 379/765] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:33.100645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944613d-5981-4767-b19c-0f17679049c2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.101353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc931c1b-d8c0-4d83-92cd-d49436fe7044 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:33.115379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 387/766] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:33.116746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc931c1b-d8c0-4d83-92cd-d49436fe7044 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.119464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:33.230155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc931c1b-d8c0-4d83-92cd-d49436fe7044 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:33.236814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944613d-5981-4767-b19c-0f17679049c2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:33.271450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4f5e2-7f29-4fa7-952e-2fe9a79fc28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:33.272598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4f5e2-7f29-4fa7-952e-2fe9a79fc28b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:33.273058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4f5e2-7f29-4fa7-952e-2fe9a79fc28b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:33.273697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4f5e2-7f29-4fa7-952e-2fe9a79fc28b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.274845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4f5e2-7f29-4fa7-952e-2fe9a79fc28b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:33.277475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4f5e2-7f29-4fa7-952e-2fe9a79fc28b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:33.280543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bf472-eb46-452f-a9e7-4b13c5f22a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:33.281067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bf472-eb46-452f-a9e7-4b13c5f22a2c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:33.281446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bf472-eb46-452f-a9e7-4b13c5f22a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:33.281896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bf472-eb46-452f-a9e7-4b13c5f22a2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e1a4f5e2-7f29-4fa7-952e-2fe9a79fc28b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:33.288108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:33.289361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 15:10:33.290409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:33.292113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bf472-eb46-452f-a9e7-4b13c5f22a2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:33.293364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 388/767] 10.222.0.75 () {56 vars in 1129 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:33.293870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bf472-eb46-452f-a9e7-4b13c5f22a2c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:33.294472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4f5e2-7f29-4fa7-952e-2fe9a79fc28b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-981bf472-eb46-452f-a9e7-4b13c5f22a2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:33.306130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:33.307489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:33.308672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:33.308672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:33.308672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:33.308672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:33.308672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:33.308672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:33.308672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:33.308672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:33.310131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 380/768] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:33.311053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bf472-eb46-452f-a9e7-4b13c5f22a2c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.322818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:33.375367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:33.418097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4f5e2-7f29-4fa7-952e-2fe9a79fc28b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:33.420563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bf472-eb46-452f-a9e7-4b13c5f22a2c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:33.453082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f5a1b-d72a-4341-8637-ff0b5a87b886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:33.453684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f5a1b-d72a-4341-8637-ff0b5a87b886 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:33.453960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f5a1b-d72a-4341-8637-ff0b5a87b886 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:33.454228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f5a1b-d72a-4341-8637-ff0b5a87b886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.454945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f5a1b-d72a-4341-8637-ff0b5a87b886 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=4d4a71b6005343cda93a92465067a7d7) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:33.455252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f5a1b-d72a-4341-8637-ff0b5a87b886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:33.456426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f512be6-7f9a-4ba1-80e0-1f3f7e2d8729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:33.456711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f5a1b-d72a-4341-8637-ff0b5a87b886 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:33.457337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f512be6-7f9a-4ba1-80e0-1f3f7e2d8729 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:33.457763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f512be6-7f9a-4ba1-80e0-1f3f7e2d8729 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:33.458179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f512be6-7f9a-4ba1-80e0-1f3f7e2d8729 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.458998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f512be6-7f9a-4ba1-80e0-1f3f7e2d8729 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:33.459449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f512be6-7f9a-4ba1-80e0-1f3f7e2d8729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:33.460841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f512be6-7f9a-4ba1-80e0-1f3f7e2d8729 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:33.472490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 389/769] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/role_assignments?user.id=4d4a71b6005343cda93a92465067a7d7&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:33.472490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f5a1b-d72a-4341-8637-ff0b5a87b886 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.472801 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 381/770] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:33.472801 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f512be6-7f9a-4ba1-80e0-1f3f7e2d8729 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.564844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:33.611696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f512be6-7f9a-4ba1-80e0-1f3f7e2d8729 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:33.652691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f5a1b-d72a-4341-8637-ff0b5a87b886 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:33.657845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c122ae-f31a-4b78-a4dc-de160d9d9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:33.658690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c122ae-f31a-4b78-a4dc-de160d9d9028 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:33.659129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c122ae-f31a-4b78-a4dc-de160d9d9028 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:33.659548 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c122ae-f31a-4b78-a4dc-de160d9d9028 None admin] PATH_INFO: `/v3/services/network` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.660277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c122ae-f31a-4b78-a4dc-de160d9d9028 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:33.662578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c122ae-f31a-4b78-a4dc-de160d9d9028 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-52c122ae-f31a-4b78-a4dc-de160d9d9028 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:33.675031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:33.676245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 15:10:33.677322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:33.680327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 382/771] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:33.681325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c122ae-f31a-4b78-a4dc-de160d9d9028 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.695252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:33.703131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef20489-6b6f-4c6d-8726-cc69ae44b823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:33.704201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef20489-6b6f-4c6d-8726-cc69ae44b823 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:33.705105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef20489-6b6f-4c6d-8726-cc69ae44b823 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:33.705624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef20489-6b6f-4c6d-8726-cc69ae44b823 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.706786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef20489-6b6f-4c6d-8726-cc69ae44b823 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:33.707465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef20489-6b6f-4c6d-8726-cc69ae44b823 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:33.709458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef20489-6b6f-4c6d-8726-cc69ae44b823 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:33.725914 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 390/772] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:33.728003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef20489-6b6f-4c6d-8726-cc69ae44b823 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.809642 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c122ae-f31a-4b78-a4dc-de160d9d9028 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:33.847544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0e2db-9761-4f88-8382-3c6254f5b6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:33.848525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0e2db-9761-4f88-8382-3c6254f5b6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:33.849040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0e2db-9761-4f88-8382-3c6254f5b6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:33.849544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0e2db-9761-4f88-8382-3c6254f5b6e8 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.850387 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0e2db-9761-4f88-8382-3c6254f5b6e8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:33.850897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0e2db-9761-4f88-8382-3c6254f5b6e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:33.852603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0e2db-9761-4f88-8382-3c6254f5b6e8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:33.855600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:33.864102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 383/773] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:33.865414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0e2db-9761-4f88-8382-3c6254f5b6e8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.868824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef20489-6b6f-4c6d-8726-cc69ae44b823 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:33.914883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc44e88-245e-4c0c-b60b-eb013d5ed5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:33.916817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc44e88-245e-4c0c-b60b-eb013d5ed5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:33.920639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc44e88-245e-4c0c-b60b-eb013d5ed5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:33.921627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc44e88-245e-4c0c-b60b-eb013d5ed5b0 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:33.922529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc44e88-245e-4c0c-b60b-eb013d5ed5b0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:33.923050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc44e88-245e-4c0c-b60b-eb013d5ed5b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:33.925467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc44e88-245e-4c0c-b60b-eb013d5ed5b0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:33.933672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 391/774] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:33.936510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc44e88-245e-4c0c-b60b-eb013d5ed5b0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:33.988929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0e2db-9761-4f88-8382-3c6254f5b6e8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:34.000596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:34.028954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730938e2-be07-4698-975b-12e409b18e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:34.029561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730938e2-be07-4698-975b-12e409b18e0b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:34.029846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730938e2-be07-4698-975b-12e409b18e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:34.030121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730938e2-be07-4698-975b-12e409b18e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:34.030865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730938e2-be07-4698-975b-12e409b18e0b None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=383ff491387642a3ab45ab692f94189c) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:34.031175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730938e2-be07-4698-975b-12e409b18e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:34.032686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730938e2-be07-4698-975b-12e409b18e0b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:34.038362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 384/775] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/role_assignments?user.id=383ff491387642a3ab45ab692f94189c&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:34.039508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0934f-a307-409b-bc85-3340c501f058 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:34.039800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0934f-a307-409b-bc85-3340c501f058 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:34.040075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0934f-a307-409b-bc85-3340c501f058 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:34.044497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 385/776] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:34.044497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0934f-a307-409b-bc85-3340c501f058 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:34.064140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc44e88-245e-4c0c-b60b-eb013d5ed5b0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:34.106791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14dd4e1-095c-4b8f-9296-102281b54b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:34.107854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dd4e1-095c-4b8f-9296-102281b54b27 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:34.108344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dd4e1-095c-4b8f-9296-102281b54b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:34.108846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dd4e1-095c-4b8f-9296-102281b54b27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:34.109862 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14dd4e1-095c-4b8f-9296-102281b54b27 None admin] RBAC: Adding query filter params (interface=public, service_id=72955deba1384e6a955634569a130a33, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:34.110472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14dd4e1-095c-4b8f-9296-102281b54b27 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:34.112205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14dd4e1-095c-4b8f-9296-102281b54b27 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:34.122849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 392/777] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Aug 19 15:10:33 2022] GET /identity/v3/endpoints?service_id=72955deba1384e6a955634569a130a33&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:34.124622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fac41c-49a0-4250-86f4-6a335a20a756 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:34.125142 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fac41c-49a0-4250-86f4-6a335a20a756 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:34.125614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fac41c-49a0-4250-86f4-6a335a20a756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:34.138508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:34.148315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c0fac41c-49a0-4250-86f4-6a335a20a756 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:34.149983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c0fac41c-49a0-4250-86f4-6a335a20a756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:34.187658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0934f-a307-409b-bc85-3340c501f058 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:34.226766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf299dfd-be40-4b3f-b4e6-385cc2ea788c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:34.227369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf299dfd-be40-4b3f-b4e6-385cc2ea788c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:34.227649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf299dfd-be40-4b3f-b4e6-385cc2ea788c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:34.227930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf299dfd-be40-4b3f-b4e6-385cc2ea788c None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:34.229152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf299dfd-be40-4b3f-b4e6-385cc2ea788c None admin] RBAC: Adding query filter params (type=network) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:34.229519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf299dfd-be40-4b3f-b4e6-385cc2ea788c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:34.231220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf299dfd-be40-4b3f-b4e6-385cc2ea788c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:34.236706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 386/778] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:34.240739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 393/779] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:34.245397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf299dfd-be40-4b3f-b4e6-385cc2ea788c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:34.251052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fac41c-49a0-4250-86f4-6a335a20a756 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:34.344506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:34.408911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf299dfd-be40-4b3f-b4e6-385cc2ea788c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:34.440666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:34.450605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c158630-0d79-4b85-aca4-7bf4cf9ca0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:34.451471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c158630-0d79-4b85-aca4-7bf4cf9ca0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:34.451936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c158630-0d79-4b85-aca4-7bf4cf9ca0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:34.452409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c158630-0d79-4b85-aca4-7bf4cf9ca0e2 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:34.453207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c158630-0d79-4b85-aca4-7bf4cf9ca0e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:34.454825 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c158630-0d79-4b85-aca4-7bf4cf9ca0e2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:34.461032 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 387/780] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Aug 19 15:10:34 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:34.468519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fac41c-49a0-4250-86f4-6a335a20a756 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:34.504863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65435a-9f59-48c7-8f6f-18e83df0979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:34.505507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65435a-9f59-48c7-8f6f-18e83df0979d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:34.505785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65435a-9f59-48c7-8f6f-18e83df0979d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:34.506053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65435a-9f59-48c7-8f6f-18e83df0979d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:34.506786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65435a-9f59-48c7-8f6f-18e83df0979d None admin] RBAC: Adding query filter params (interface=public, service_id=978a3cf102904b0d85d34cbad85820bf, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:34.507091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65435a-9f59-48c7-8f6f-18e83df0979d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:34.508541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65435a-9f59-48c7-8f6f-18e83df0979d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:34.515349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 394/781] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Aug 19 15:10:34 2022] GET /identity/v3/endpoints?service_id=978a3cf102904b0d85d34cbad85820bf&interface=public®ion_id=RegionOne => generated 194 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:34.520615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c158630-0d79-4b85-aca4-7bf4cf9ca0e2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:34.626397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65435a-9f59-48c7-8f6f-18e83df0979d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:34.662551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c158630-0d79-4b85-aca4-7bf4cf9ca0e2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:34.710463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ba478-3885-4969-a920-c51aa24a1005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:34.711385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ba478-3885-4969-a920-c51aa24a1005 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:34.713509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ba478-3885-4969-a920-c51aa24a1005 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:34.714137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ba478-3885-4969-a920-c51aa24a1005 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:34.716360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ba478-3885-4969-a920-c51aa24a1005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:34.718701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ba478-3885-4969-a920-c51aa24a1005 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:34.726018 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 388/782] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Aug 19 15:10:34 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:34.770476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65435a-9f59-48c7-8f6f-18e83df0979d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:34.788091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:34.813540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6821-1bf8-4450-89f0-e1c650e169cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:34.814570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6821-1bf8-4450-89f0-e1c650e169cc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:34.815060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6821-1bf8-4450-89f0-e1c650e169cc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:34.815535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6821-1bf8-4450-89f0-e1c650e169cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:34.828426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6821-1bf8-4450-89f0-e1c650e169cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5e5d6821-1bf8-4450-89f0-e1c650e169cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:34.833722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:34.834708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:34.835576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:34.842040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 395/783] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:34.845472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ba478-3885-4969-a920-c51aa24a1005 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:34.936419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:34.985079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ba478-3885-4969-a920-c51aa24a1005 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:35.028077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981c0ad-ee2b-44a7-9dc0-ea1f1dd788c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:35.028762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981c0ad-ee2b-44a7-9dc0-ea1f1dd788c3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:35.029040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981c0ad-ee2b-44a7-9dc0-ea1f1dd788c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:35.029307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981c0ad-ee2b-44a7-9dc0-ea1f1dd788c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:35.029980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981c0ad-ee2b-44a7-9dc0-ea1f1dd788c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:35.030277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981c0ad-ee2b-44a7-9dc0-ea1f1dd788c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:35.031658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981c0ad-ee2b-44a7-9dc0-ea1f1dd788c3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:35.040374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 389/784] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:35.046727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6821-1bf8-4450-89f0-e1c650e169cc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:35.161952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6821-1bf8-4450-89f0-e1c650e169cc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:35.198976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe26aa1-6857-41be-842a-66508ffbc3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:35.200132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe26aa1-6857-41be-842a-66508ffbc3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:35.200647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe26aa1-6857-41be-842a-66508ffbc3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:35.201128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe26aa1-6857-41be-842a-66508ffbc3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:35.212805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe26aa1-6857-41be-842a-66508ffbc3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:35.214784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe26aa1-6857-41be-842a-66508ffbc3c0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9fe26aa1-6857-41be-842a-66508ffbc3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:35.229897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:35.230858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:35.231783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:35.231783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:35.231783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:35.238264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 396/785] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:35.241471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981c0ad-ee2b-44a7-9dc0-ea1f1dd788c3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:35.390416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981c0ad-ee2b-44a7-9dc0-ea1f1dd788c3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:35.410141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:35.432973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba172d61-a71a-4f9c-b866-49d91cb8c997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:35.433667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba172d61-a71a-4f9c-b866-49d91cb8c997 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:35.433948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba172d61-a71a-4f9c-b866-49d91cb8c997 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:35.434221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba172d61-a71a-4f9c-b866-49d91cb8c997 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:35.434870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba172d61-a71a-4f9c-b866-49d91cb8c997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:35.435169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba172d61-a71a-4f9c-b866-49d91cb8c997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:35.436687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba172d61-a71a-4f9c-b866-49d91cb8c997 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:35.453775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 390/786] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:35.459774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe26aa1-6857-41be-842a-66508ffbc3c0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:35.508551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:35.542366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e4cac-cd86-4096-85fe-9411ce11b269 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:35.542684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e4cac-cd86-4096-85fe-9411ce11b269 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:35.542913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e4cac-cd86-4096-85fe-9411ce11b269 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:35.544513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 391/787] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:10:35.549365 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805bb08-6e76-44ae-8f99-343b38856f1f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:35.549661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805bb08-6e76-44ae-8f99-343b38856f1f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:35.549914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805bb08-6e76-44ae-8f99-343b38856f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:35.569120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f805bb08-6e76-44ae-8f99-343b38856f1f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:35.570310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f805bb08-6e76-44ae-8f99-343b38856f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:35.634102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe26aa1-6857-41be-842a-66508ffbc3c0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:35.692742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 392/788] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:35.699316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e6bc5-3fc2-47d2-8c72-5887c0f3d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:35.700022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e6bc5-3fc2-47d2-8c72-5887c0f3d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:35.700300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e6bc5-3fc2-47d2-8c72-5887c0f3d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:35.700635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e6bc5-3fc2-47d2-8c72-5887c0f3d2b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:35.713549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e6bc5-3fc2-47d2-8c72-5887c0f3d2b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:35.715175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e6bc5-3fc2-47d2-8c72-5887c0f3d2b0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e45e6bc5-3fc2-47d2-8c72-5887c0f3d2b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:35.732864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:35.733850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:35.735215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 397/789] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Aug 19 15:10:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:35.741751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805bb08-6e76-44ae-8f99-343b38856f1f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:35.864352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805bb08-6e76-44ae-8f99-343b38856f1f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:35.874840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:35.911906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53670b34-a7c5-4708-bd6c-c051295ccd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:35.912534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53670b34-a7c5-4708-bd6c-c051295ccd97 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:35.912812 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53670b34-a7c5-4708-bd6c-c051295ccd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:35.913137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53670b34-a7c5-4708-bd6c-c051295ccd97 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:35.913870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53670b34-a7c5-4708-bd6c-c051295ccd97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:35.914170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53670b34-a7c5-4708-bd6c-c051295ccd97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:35.915708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53670b34-a7c5-4708-bd6c-c051295ccd97 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:35.942085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 393/790] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Aug 19 15:10:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:35.948344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e6bc5-3fc2-47d2-8c72-5887c0f3d2b0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:35.984810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53670b34-a7c5-4708-bd6c-c051295ccd97 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:36.089656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:36.121223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e6bc5-3fc2-47d2-8c72-5887c0f3d2b0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:36.136537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53670b34-a7c5-4708-bd6c-c051295ccd97 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:36.177828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd13c73-6388-47d2-8f32-00f6547f5ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:36.178449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd13c73-6388-47d2-8f32-00f6547f5ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:36.178730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd13c73-6388-47d2-8f32-00f6547f5ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:36.179007 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd13c73-6388-47d2-8f32-00f6547f5ce5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:36.207305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f28b8-d1ea-4ece-8a78-6fe43e32edd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:36.209413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f28b8-d1ea-4ece-8a78-6fe43e32edd7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:36.209698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f28b8-d1ea-4ece-8a78-6fe43e32edd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:36.209972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f28b8-d1ea-4ece-8a78-6fe43e32edd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:36.231701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f28b8-d1ea-4ece-8a78-6fe43e32edd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-300f28b8-d1ea-4ece-8a78-6fe43e32edd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:36.236534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:36.237479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:36.238328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:36.238971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 394/791] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:36.244863 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f28b8-d1ea-4ece-8a78-6fe43e32edd7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:36.300480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:36.322725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd13c73-6388-47d2-8f32-00f6547f5ce5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:36.324190 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd13c73-6388-47d2-8f32-00f6547f5ce5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:36.432887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:36.449704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 398/792] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:10:36.456051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd13c73-6388-47d2-8f32-00f6547f5ce5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:36.467419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f28b8-d1ea-4ece-8a78-6fe43e32edd7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:36.520135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ed02e-9796-4fa8-8830-72fcf7c38a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:36.520886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ed02e-9796-4fa8-8830-72fcf7c38a38 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:36.521262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ed02e-9796-4fa8-8830-72fcf7c38a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:36.521607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ed02e-9796-4fa8-8830-72fcf7c38a38 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:36.522552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ed02e-9796-4fa8-8830-72fcf7c38a38 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:36.522712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ed02e-9796-4fa8-8830-72fcf7c38a38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:36.524220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ed02e-9796-4fa8-8830-72fcf7c38a38 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:36.538785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 395/793] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:36.540069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ed02e-9796-4fa8-8830-72fcf7c38a38 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:36.585110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd13c73-6388-47d2-8f32-00f6547f5ce5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:36.625341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1428d-890f-438d-bf5c-c99ac62bcac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:36.626278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1428d-890f-438d-bf5c-c99ac62bcac5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:36.626831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1428d-890f-438d-bf5c-c99ac62bcac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:36.627307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1428d-890f-438d-bf5c-c99ac62bcac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:36.634396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:36.641915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1428d-890f-438d-bf5c-c99ac62bcac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:36.643758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1428d-890f-438d-bf5c-c99ac62bcac5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:36.648877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ed02e-9796-4fa8-8830-72fcf7c38a38 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0cf1428d-890f-438d-bf5c-c99ac62bcac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:36.660328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:36.661363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:36.662271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:36.662271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:36.662271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:36.664493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 399/794] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:36.665916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2644b-eb00-450b-b3c4-e4c5c43e2339 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:36.666420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2644b-eb00-450b-b3c4-e4c5c43e2339 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:36.666854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2644b-eb00-450b-b3c4-e4c5c43e2339 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:36.672848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 400/795] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:36.674369 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b681915-fc81-4d8d-964f-5af8e419612a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:36.674843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b681915-fc81-4d8d-964f-5af8e419612a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:36.675278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b681915-fc81-4d8d-964f-5af8e419612a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:36.698462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0b681915-fc81-4d8d-964f-5af8e419612a None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:36.699980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0b681915-fc81-4d8d-964f-5af8e419612a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:36.705079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae4a1a-5b1b-42cc-8228-8fac04334793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:36.705898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae4a1a-5b1b-42cc-8228-8fac04334793 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:36.706259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae4a1a-5b1b-42cc-8228-8fac04334793 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:36.706606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae4a1a-5b1b-42cc-8228-8fac04334793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:36.719070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae4a1a-5b1b-42cc-8228-8fac04334793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:36.720702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae4a1a-5b1b-42cc-8228-8fac04334793 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a7ae4a1a-5b1b-42cc-8228-8fac04334793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:36.736607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:36.737592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:36.738557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:36.738557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:36.738557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:36.738741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:36.740474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 396/796] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:36.741547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae4a1a-5b1b-42cc-8228-8fac04334793 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:36.788630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 401/797] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:36.790006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b681915-fc81-4d8d-964f-5af8e419612a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:36.852323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae4a1a-5b1b-42cc-8228-8fac04334793 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:36.889677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db08729-9f62-4d8e-9460-9250db83217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:36.890339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db08729-9f62-4d8e-9460-9250db83217e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:36.890622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db08729-9f62-4d8e-9460-9250db83217e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:36.890898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db08729-9f62-4d8e-9460-9250db83217e None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:36.891715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db08729-9f62-4d8e-9460-9250db83217e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:36.892083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db08729-9f62-4d8e-9460-9250db83217e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:36.893843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db08729-9f62-4d8e-9460-9250db83217e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:36.908431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:36.914437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 397/798] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:36.915781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db08729-9f62-4d8e-9460-9250db83217e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.020422 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b681915-fc81-4d8d-964f-5af8e419612a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.038613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db08729-9f62-4d8e-9460-9250db83217e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.069665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca585fc-20a1-4787-964b-4483b65cecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.070338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca585fc-20a1-4787-964b-4483b65cecb3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.070618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca585fc-20a1-4787-964b-4483b65cecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.071130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca585fc-20a1-4787-964b-4483b65cecb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.072179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca585fc-20a1-4787-964b-4483b65cecb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:37.073028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca585fc-20a1-4787-964b-4483b65cecb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:37.074987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca585fc-20a1-4787-964b-4483b65cecb3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:37.080703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f3ed-6011-4e81-9e38-bf5578a4ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.081413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f3ed-6011-4e81-9e38-bf5578a4ec40 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.081767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f3ed-6011-4e81-9e38-bf5578a4ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.082129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f3ed-6011-4e81-9e38-bf5578a4ec40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.089059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 402/799] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:37.090314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d89023-5a64-498e-ad76-0e93a1cbf9c0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.090597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d89023-5a64-498e-ad76-0e93a1cbf9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.090877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d89023-5a64-498e-ad76-0e93a1cbf9c0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.096504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 403/800] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:37.097124 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513f3ed-6011-4e81-9e38-bf5578a4ec40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:37.097291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d89023-5a64-498e-ad76-0e93a1cbf9c0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.099044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513f3ed-6011-4e81-9e38-bf5578a4ec40 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9513f3ed-6011-4e81-9e38-bf5578a4ec40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:37.112495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:37.113531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:37.116584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:37.116584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:37.116584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:37.116584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:37.116584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:37.116584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:37.116584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:37.116584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:37.118218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 398/801] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:37.119478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2764f4-1578-4477-a31a-d89c38ae272f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.119759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2764f4-1578-4477-a31a-d89c38ae272f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.120037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2764f4-1578-4477-a31a-d89c38ae272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.128350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:37.138339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ce2764f4-1578-4477-a31a-d89c38ae272f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:37.140054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ce2764f4-1578-4477-a31a-d89c38ae272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:37.199110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:37.209809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d89023-5a64-498e-ad76-0e93a1cbf9c0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.210748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 399/802] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:37.211561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2764f4-1578-4477-a31a-d89c38ae272f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.253632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97279afe-b320-4619-8119-ca0fb78d2216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.254261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97279afe-b320-4619-8119-ca0fb78d2216 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.254537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97279afe-b320-4619-8119-ca0fb78d2216 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.254808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97279afe-b320-4619-8119-ca0fb78d2216 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.263015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97279afe-b320-4619-8119-ca0fb78d2216 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:37.264509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97279afe-b320-4619-8119-ca0fb78d2216 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-97279afe-b320-4619-8119-ca0fb78d2216 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:37.276022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:37.277311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 404/803] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:10:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:37.278251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97279afe-b320-4619-8119-ca0fb78d2216 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.331298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2764f4-1578-4477-a31a-d89c38ae272f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.369847 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6becd6f0-dfc7-4e5e-aad2-83e30c0637c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.370606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becd6f0-dfc7-4e5e-aad2-83e30c0637c4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.370880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becd6f0-dfc7-4e5e-aad2-83e30c0637c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.371105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:37.371224 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becd6f0-dfc7-4e5e-aad2-83e30c0637c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.383212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6becd6f0-dfc7-4e5e-aad2-83e30c0637c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6becd6f0-dfc7-4e5e-aad2-83e30c0637c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:37.388619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:37.389886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 400/804] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:37.390913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa62ffc-acb8-4a70-8570-f902b727a836 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.391667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa62ffc-acb8-4a70-8570-f902b727a836 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.391667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa62ffc-acb8-4a70-8570-f902b727a836 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.392816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 401/805] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:37.393390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa62ffc-acb8-4a70-8570-f902b727a836 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.400597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97279afe-b320-4619-8119-ca0fb78d2216 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.438951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe14ba3-dee0-45ad-b2b4-312acaf63984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.439641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe14ba3-dee0-45ad-b2b4-312acaf63984 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.439924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe14ba3-dee0-45ad-b2b4-312acaf63984 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.440193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe14ba3-dee0-45ad-b2b4-312acaf63984 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.440908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe14ba3-dee0-45ad-b2b4-312acaf63984 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:37.441213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe14ba3-dee0-45ad-b2b4-312acaf63984 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:37.442655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe14ba3-dee0-45ad-b2b4-312acaf63984 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:37.450843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:37.458698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 405/806] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:37.459911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe14ba3-dee0-45ad-b2b4-312acaf63984 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.507506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa62ffc-acb8-4a70-8570-f902b727a836 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.543273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1278d6-5304-4e2c-aae5-dafbb41752a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.543870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1278d6-5304-4e2c-aae5-dafbb41752a3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.544146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1278d6-5304-4e2c-aae5-dafbb41752a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.544438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1278d6-5304-4e2c-aae5-dafbb41752a3 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.545164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1278d6-5304-4e2c-aae5-dafbb41752a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:37.545468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1278d6-5304-4e2c-aae5-dafbb41752a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:37.546933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1278d6-5304-4e2c-aae5-dafbb41752a3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:37.565769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe14ba3-dee0-45ad-b2b4-312acaf63984 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.567482 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 402/807] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Aug 19 15:10:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:37.568728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3d22-a96b-42e0-9253-95e61c645a3c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.569017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3d22-a96b-42e0-9253-95e61c645a3c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.569290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3d22-a96b-42e0-9253-95e61c645a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.587069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-370e3d22-a96b-42e0-9253-95e61c645a3c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:37.588455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-370e3d22-a96b-42e0-9253-95e61c645a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:37.606061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e60738-6537-4f78-8a00-6dc7854e3d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.606664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e60738-6537-4f78-8a00-6dc7854e3d48 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.606940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e60738-6537-4f78-8a00-6dc7854e3d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.607211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e60738-6537-4f78-8a00-6dc7854e3d48 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.607871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e60738-6537-4f78-8a00-6dc7854e3d48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:37.608171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e60738-6537-4f78-8a00-6dc7854e3d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:37.609589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e60738-6537-4f78-8a00-6dc7854e3d48 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:37.617796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 406/808] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:37.618568 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e60738-6537-4f78-8a00-6dc7854e3d48 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.629666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:37.657270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 403/809] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:37.658110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e3d22-a96b-42e0-9253-95e61c645a3c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.711633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:37.729421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e60738-6537-4f78-8a00-6dc7854e3d48 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.771064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82327e25-8a60-4c35-913f-e66eb780eeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.772123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82327e25-8a60-4c35-913f-e66eb780eeba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.772436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82327e25-8a60-4c35-913f-e66eb780eeba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.772778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82327e25-8a60-4c35-913f-e66eb780eeba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.773467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82327e25-8a60-4c35-913f-e66eb780eeba None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=98a63d5b605a4ecfad2e305c491ae755) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:37.773783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82327e25-8a60-4c35-913f-e66eb780eeba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:37.775230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82327e25-8a60-4c35-913f-e66eb780eeba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:37.780711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e3d22-a96b-42e0-9253-95e61c645a3c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.784228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 407/810] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:37 2022] GET /identity/v3/role_assignments?user.id=98a63d5b605a4ecfad2e305c491ae755&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:37.785361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82327e25-8a60-4c35-913f-e66eb780eeba None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.831232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3be1f-5390-4800-9ecd-7e66b839c8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.831931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3be1f-5390-4800-9ecd-7e66b839c8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.832239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3be1f-5390-4800-9ecd-7e66b839c8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.832559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3be1f-5390-4800-9ecd-7e66b839c8b1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.833158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3be1f-5390-4800-9ecd-7e66b839c8b1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:37.834581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3be1f-5390-4800-9ecd-7e66b839c8b1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8ec3be1f-5390-4800-9ecd-7e66b839c8b1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:37.845608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:37.846660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 15:10:37.847604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:37.851699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 404/811] 10.222.0.75 () {56 vars in 1129 bytes} [Fri Aug 19 15:10:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:37.852737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3be1f-5390-4800-9ecd-7e66b839c8b1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:37.890958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:37.926953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82327e25-8a60-4c35-913f-e66eb780eeba None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.969521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4c6f1-5f9a-4672-9229-61f59762294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:37.970230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4c6f1-5f9a-4672-9229-61f59762294f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:37.970605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4c6f1-5f9a-4672-9229-61f59762294f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:37.970938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4c6f1-5f9a-4672-9229-61f59762294f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:37.980595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3be1f-5390-4800-9ecd-7e66b839c8b1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:37.982385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:37.986313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4c6f1-5f9a-4672-9229-61f59762294f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:37.987788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4c6f1-5f9a-4672-9229-61f59762294f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b9b4c6f1-5f9a-4672-9229-61f59762294f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:38.006718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:38.007753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:38.009709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:38.009709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:38.009709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:38.009709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 408/812] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:38.009709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4c6f1-5f9a-4672-9229-61f59762294f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.024055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c046bc-8bd8-4a78-85b3-e3a15b6f02b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.024707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c046bc-8bd8-4a78-85b3-e3a15b6f02b3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.024992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c046bc-8bd8-4a78-85b3-e3a15b6f02b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.025263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c046bc-8bd8-4a78-85b3-e3a15b6f02b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.036470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c046bc-8bd8-4a78-85b3-e3a15b6f02b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.037933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c046bc-8bd8-4a78-85b3-e3a15b6f02b3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-98c046bc-8bd8-4a78-85b3-e3a15b6f02b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:38.052501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:38.053626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:38.054691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:38.054691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:38.054691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:38.054691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 405/813] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:38.058091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c046bc-8bd8-4a78-85b3-e3a15b6f02b3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.123178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4c6f1-5f9a-4672-9229-61f59762294f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.147012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:38.159967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd9d2f-44bb-405e-833f-9770db762e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.160870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd9d2f-44bb-405e-833f-9770db762e9a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.160870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd9d2f-44bb-405e-833f-9770db762e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.161169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd9d2f-44bb-405e-833f-9770db762e9a None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.161792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd9d2f-44bb-405e-833f-9770db762e9a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:38.162108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd9d2f-44bb-405e-833f-9770db762e9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.163476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd9d2f-44bb-405e-833f-9770db762e9a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.174163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 409/814] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:38.174163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd9d2f-44bb-405e-833f-9770db762e9a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.174440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c046bc-8bd8-4a78-85b3-e3a15b6f02b3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.212170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cb0fe-ef6a-4c33-a830-886a3f104401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.213456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cb0fe-ef6a-4c33-a830-886a3f104401 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.213959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cb0fe-ef6a-4c33-a830-886a3f104401 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.214239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cb0fe-ef6a-4c33-a830-886a3f104401 None admin] PATH_INFO: `/v3/services/network` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.214826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cb0fe-ef6a-4c33-a830-886a3f104401 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.216216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cb0fe-ef6a-4c33-a830-886a3f104401 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4b8cb0fe-ef6a-4c33-a830-886a3f104401 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:38.225684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:38.226691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:38.227557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 406/815] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:38.228378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cb0fe-ef6a-4c33-a830-886a3f104401 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.252706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:38.288256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd9d2f-44bb-405e-833f-9770db762e9a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.324047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a9eaa-3be8-4cab-a28f-c7113cd37bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.324699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a9eaa-3be8-4cab-a28f-c7113cd37bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.324979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a9eaa-3be8-4cab-a28f-c7113cd37bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.325254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a9eaa-3be8-4cab-a28f-c7113cd37bc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.325899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a9eaa-3be8-4cab-a28f-c7113cd37bc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:38.326203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a9eaa-3be8-4cab-a28f-c7113cd37bc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.327639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a9eaa-3be8-4cab-a28f-c7113cd37bc7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.341385 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 410/816] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:38.342274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a9eaa-3be8-4cab-a28f-c7113cd37bc7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.343270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cb0fe-ef6a-4c33-a830-886a3f104401 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.382390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06964478-ed48-46ad-a415-924cfec9ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.382918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06964478-ed48-46ad-a415-924cfec9ef68 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.383199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06964478-ed48-46ad-a415-924cfec9ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.383474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06964478-ed48-46ad-a415-924cfec9ef68 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.384125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06964478-ed48-46ad-a415-924cfec9ef68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:38.384456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06964478-ed48-46ad-a415-924cfec9ef68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.385897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06964478-ed48-46ad-a415-924cfec9ef68 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.403626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 407/817] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:38.403626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06964478-ed48-46ad-a415-924cfec9ef68 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.407800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:38.460903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a9eaa-3be8-4cab-a28f-c7113cd37bc7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.489608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:38.499429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc90643-ba5c-42d9-b685-d437e958d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.500042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc90643-ba5c-42d9-b685-d437e958d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.500315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc90643-ba5c-42d9-b685-d437e958d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.500657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc90643-ba5c-42d9-b685-d437e958d3c1 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.501272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc90643-ba5c-42d9-b685-d437e958d3c1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:38.501572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc90643-ba5c-42d9-b685-d437e958d3c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.503047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc90643-ba5c-42d9-b685-d437e958d3c1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.508064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 411/818] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:38.513567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc90643-ba5c-42d9-b685-d437e958d3c1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.513809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06964478-ed48-46ad-a415-924cfec9ef68 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.549597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717896f-c92b-41de-9ea6-7a9e47603fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.550832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717896f-c92b-41de-9ea6-7a9e47603fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.551016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717896f-c92b-41de-9ea6-7a9e47603fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.551328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717896f-c92b-41de-9ea6-7a9e47603fb6 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.551874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717896f-c92b-41de-9ea6-7a9e47603fb6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:38.552424 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717896f-c92b-41de-9ea6-7a9e47603fb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.553581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717896f-c92b-41de-9ea6-7a9e47603fb6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.558562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 408/819] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:38.559178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717896f-c92b-41de-9ea6-7a9e47603fb6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.628403 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc90643-ba5c-42d9-b685-d437e958d3c1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.643721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:38.667572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717896f-c92b-41de-9ea6-7a9e47603fb6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.669563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8519436-51a4-4e31-9f6d-c3f59a0b084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.670863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8519436-51a4-4e31-9f6d-c3f59a0b084d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.671660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8519436-51a4-4e31-9f6d-c3f59a0b084d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.672260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8519436-51a4-4e31-9f6d-c3f59a0b084d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.683840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8519436-51a4-4e31-9f6d-c3f59a0b084d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.685454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8519436-51a4-4e31-9f6d-c3f59a0b084d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c8519436-51a4-4e31-9f6d-c3f59a0b084d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:38.697839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:38.698971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:38.699877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:38.699877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:38.699877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:38.699877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:38.699877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:38.699877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:38.699877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:38.699877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:38.703833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 412/820] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:38.704674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8519436-51a4-4e31-9f6d-c3f59a0b084d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.711832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d2701-2069-45d7-94c1-bf2a007a6151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.712493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d2701-2069-45d7-94c1-bf2a007a6151 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.712779 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d2701-2069-45d7-94c1-bf2a007a6151 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.713710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d2701-2069-45d7-94c1-bf2a007a6151 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.724840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d2701-2069-45d7-94c1-bf2a007a6151 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.726310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d2701-2069-45d7-94c1-bf2a007a6151 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-641d2701-2069-45d7-94c1-bf2a007a6151 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:38.740780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:38.741767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:38.742714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 409/821] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:38.743310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d2701-2069-45d7-94c1-bf2a007a6151 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.746213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:38.827651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8519436-51a4-4e31-9f6d-c3f59a0b084d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.855900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d2701-2069-45d7-94c1-bf2a007a6151 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:38.864567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4d630-f32a-4e25-bc40-fccba6af21b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.865188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4d630-f32a-4e25-bc40-fccba6af21b6 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.865466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4d630-f32a-4e25-bc40-fccba6af21b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.865741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4d630-f32a-4e25-bc40-fccba6af21b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.866401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4d630-f32a-4e25-bc40-fccba6af21b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.867789 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4d630-f32a-4e25-bc40-fccba6af21b6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.900552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 413/822] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:10:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:38.900552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4d630-f32a-4e25-bc40-fccba6af21b6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.909858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a93f6e-d497-4268-9b6d-a73792d79832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:38.910656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a93f6e-d497-4268-9b6d-a73792d79832 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:38.911004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a93f6e-d497-4268-9b6d-a73792d79832 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:38.911387 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a93f6e-d497-4268-9b6d-a73792d79832 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:38.912237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a93f6e-d497-4268-9b6d-a73792d79832 None admin] RBAC: Adding query filter params (type=network) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:38.912598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a93f6e-d497-4268-9b6d-a73792d79832 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:38.914175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a93f6e-d497-4268-9b6d-a73792d79832 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:38.920614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 410/823] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:38.921545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a93f6e-d497-4268-9b6d-a73792d79832 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:38.947025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:39.030828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:39.053181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4d630-f32a-4e25-bc40-fccba6af21b6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:39.097699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee6d0d-6ce6-48c8-98a2-13d00a27dab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:39.098323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee6d0d-6ce6-48c8-98a2-13d00a27dab0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:39.098596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee6d0d-6ce6-48c8-98a2-13d00a27dab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:39.098867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee6d0d-6ce6-48c8-98a2-13d00a27dab0 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:39.099571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee6d0d-6ce6-48c8-98a2-13d00a27dab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:39.099871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee6d0d-6ce6-48c8-98a2-13d00a27dab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:39.101396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee6d0d-6ce6-48c8-98a2-13d00a27dab0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:39.116705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 414/824] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:39.117363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee6d0d-6ce6-48c8-98a2-13d00a27dab0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:39.200423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a93f6e-d497-4268-9b6d-a73792d79832 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:39.235629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:39.240292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a411d-d10b-4fff-8fc4-89e4b467a51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:39.241019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a411d-d10b-4fff-8fc4-89e4b467a51f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:39.241298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a411d-d10b-4fff-8fc4-89e4b467a51f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:39.241565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a411d-d10b-4fff-8fc4-89e4b467a51f None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:39.242304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a411d-d10b-4fff-8fc4-89e4b467a51f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:39.242888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a411d-d10b-4fff-8fc4-89e4b467a51f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:39.244019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a411d-d10b-4fff-8fc4-89e4b467a51f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:39.255564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee6d0d-6ce6-48c8-98a2-13d00a27dab0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:39.264026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 411/825] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Aug 19 15:10:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:39.264965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a411d-d10b-4fff-8fc4-89e4b467a51f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:39.309580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d604b-ad86-4cc8-9ee4-2b2416a1e206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:39.310588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d604b-ad86-4cc8-9ee4-2b2416a1e206 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:39.311084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d604b-ad86-4cc8-9ee4-2b2416a1e206 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:39.311562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d604b-ad86-4cc8-9ee4-2b2416a1e206 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:39.312568 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d604b-ad86-4cc8-9ee4-2b2416a1e206 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:39.314360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d604b-ad86-4cc8-9ee4-2b2416a1e206 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:39.344786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 415/826] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:10:39 2022] POST /identity/v3/endpoints => generated 337 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:39.346131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d604b-ad86-4cc8-9ee4-2b2416a1e206 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:39.387522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a411d-d10b-4fff-8fc4-89e4b467a51f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:39.410930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:39.449675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a88670-49e5-44f5-8761-72a3c16aa47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:39.450296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a88670-49e5-44f5-8761-72a3c16aa47f None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:39.450624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a88670-49e5-44f5-8761-72a3c16aa47f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:39.450843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a88670-49e5-44f5-8761-72a3c16aa47f None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/4d4a71b6005343cda93a92465067a7d7/roles/fc63a4da31e84ad78f04c587c0c1c83c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:39.494691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a88670-49e5-44f5-8761-72a3c16aa47f None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=4d4a71b6005343cda93a92465067a7d7, role_id=fc63a4da31e84ad78f04c587c0c1c83c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:39.496501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a88670-49e5-44f5-8761-72a3c16aa47f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:39.525019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 412/827] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:10:39 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/4d4a71b6005343cda93a92465067a7d7/roles/fc63a4da31e84ad78f04c587c0c1c83c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:10:39.663143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d604b-ad86-4cc8-9ee4-2b2416a1e206 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:39.714771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2c615-719e-447a-b42c-980ae10e146d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:39.715326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2c615-719e-447a-b42c-980ae10e146d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:39.715599 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2c615-719e-447a-b42c-980ae10e146d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:39.715882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2c615-719e-447a-b42c-980ae10e146d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:39.728382 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2c615-719e-447a-b42c-980ae10e146d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:39.730632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2c615-719e-447a-b42c-980ae10e146d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:39.734381 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0ae2c615-719e-447a-b42c-980ae10e146d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:39.750159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:39.751161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:39.752064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:39.752064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:39.752064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:39.752064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 416/828] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:39.757181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a88670-49e5-44f5-8761-72a3c16aa47f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:39.815398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5784-f6d5-4728-9a19-53d0a472c17d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:39.815731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5784-f6d5-4728-9a19-53d0a472c17d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:39.815949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5784-f6d5-4728-9a19-53d0a472c17d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:39.817479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 417/829] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:39.825002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50defa61-1b24-4abd-86e5-494977784273 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:39.825574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50defa61-1b24-4abd-86e5-494977784273 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:39.826073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50defa61-1b24-4abd-86e5-494977784273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:39.854532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-50defa61-1b24-4abd-86e5-494977784273 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:39.856806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-50defa61-1b24-4abd-86e5-494977784273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:39.894624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a88670-49e5-44f5-8761-72a3c16aa47f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:39.938205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb4659-8b8e-49fe-a893-ee72bd690c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:39.939202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb4659-8b8e-49fe-a893-ee72bd690c0c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:39.939699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb4659-8b8e-49fe-a893-ee72bd690c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:39.940176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb4659-8b8e-49fe-a893-ee72bd690c0c None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:39.941067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb4659-8b8e-49fe-a893-ee72bd690c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:39.941670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb4659-8b8e-49fe-a893-ee72bd690c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:39.943310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb4659-8b8e-49fe-a893-ee72bd690c0c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:39.943906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 418/830] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:39.956288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 413/831] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:39.965266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50defa61-1b24-4abd-86e5-494977784273 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:40.071123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50defa61-1b24-4abd-86e5-494977784273 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:40.102017 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:40.110911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6254aff-10c1-4976-ac49-53af8c8f322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:40.111549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6254aff-10c1-4976-ac49-53af8c8f322b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:40.111821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6254aff-10c1-4976-ac49-53af8c8f322b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:40.112086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6254aff-10c1-4976-ac49-53af8c8f322b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:40.118957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6254aff-10c1-4976-ac49-53af8c8f322b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:40.121154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6254aff-10c1-4976-ac49-53af8c8f322b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b6254aff-10c1-4976-ac49-53af8c8f322b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:40.132540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:40.133455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:40.134338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:40.134338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:40.134338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:40.134338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:40.134338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:40.134338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:40.134338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:40.134338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:40.134338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 419/832] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:40.141663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb4659-8b8e-49fe-a893-ee72bd690c0c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:40.172879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:40.249284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb4659-8b8e-49fe-a893-ee72bd690c0c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:40.258663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6254aff-10c1-4976-ac49-53af8c8f322b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:40.286982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58b6e4-6572-49a2-a5f4-a0af64123186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:40.287545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58b6e4-6572-49a2-a5f4-a0af64123186 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:40.287820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58b6e4-6572-49a2-a5f4-a0af64123186 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:40.288090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58b6e4-6572-49a2-a5f4-a0af64123186 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:40.288815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58b6e4-6572-49a2-a5f4-a0af64123186 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:40.289147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58b6e4-6572-49a2-a5f4-a0af64123186 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:40.290538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58b6e4-6572-49a2-a5f4-a0af64123186 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:40.303521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 414/833] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:40.310455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58b6e4-6572-49a2-a5f4-a0af64123186 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:40.376152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6254aff-10c1-4976-ac49-53af8c8f322b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:40.413698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c91011-1daf-4a52-bc8d-3a5d1b03618a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:40.414322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c91011-1daf-4a52-bc8d-3a5d1b03618a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:40.414602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c91011-1daf-4a52-bc8d-3a5d1b03618a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:40.414875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c91011-1daf-4a52-bc8d-3a5d1b03618a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:40.415458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c91011-1daf-4a52-bc8d-3a5d1b03618a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:40.417501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c91011-1daf-4a52-bc8d-3a5d1b03618a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b5c91011-1daf-4a52-bc8d-3a5d1b03618a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:40.426830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:40.427784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:40.428711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 420/834] 10.222.0.75 () {56 vars in 1132 bytes} [Fri Aug 19 15:10:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:40.435140 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:40.435680 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c91011-1daf-4a52-bc8d-3a5d1b03618a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:40.448659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58b6e4-6572-49a2-a5f4-a0af64123186 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:40.491390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:40.493973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14516b65-fbd2-4884-9d2a-6879def3691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:40.494835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14516b65-fbd2-4884-9d2a-6879def3691f None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:40.495179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14516b65-fbd2-4884-9d2a-6879def3691f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:40.496249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14516b65-fbd2-4884-9d2a-6879def3691f None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/383ff491387642a3ab45ab692f94189c/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:40.499709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14516b65-fbd2-4884-9d2a-6879def3691f None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=383ff491387642a3ab45ab692f94189c, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:40.501925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14516b65-fbd2-4884-9d2a-6879def3691f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:40.531766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 415/835] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:10:40 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/383ff491387642a3ab45ab692f94189c/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:10:40.560314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c91011-1daf-4a52-bc8d-3a5d1b03618a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:40.605473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f44fa7-d5d3-4cc5-a757-64f6862f51d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:40.606567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f44fa7-d5d3-4cc5-a757-64f6862f51d6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:40.607038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f44fa7-d5d3-4cc5-a757-64f6862f51d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:40.607484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f44fa7-d5d3-4cc5-a757-64f6862f51d6 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:40.608329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f44fa7-d5d3-4cc5-a757-64f6862f51d6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:40.608927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f44fa7-d5d3-4cc5-a757-64f6862f51d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:40.610912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f44fa7-d5d3-4cc5-a757-64f6862f51d6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:40.617409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 421/836] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:10:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:40.623807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14516b65-fbd2-4884-9d2a-6879def3691f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:40.739521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14516b65-fbd2-4884-9d2a-6879def3691f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:40.784122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33a650-02f2-4e18-8f1d-f8d25bb1dd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:40.785048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a650-02f2-4e18-8f1d-f8d25bb1dd3b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:40.785466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a650-02f2-4e18-8f1d-f8d25bb1dd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:40.785875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33a650-02f2-4e18-8f1d-f8d25bb1dd3b None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:40.786655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33a650-02f2-4e18-8f1d-f8d25bb1dd3b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:40.787119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33a650-02f2-4e18-8f1d-f8d25bb1dd3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:40.788651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33a650-02f2-4e18-8f1d-f8d25bb1dd3b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:40.794478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 416/837] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:10:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:41.157225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544bab-6721-4e20-988e-52d00c254f01 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:41.158103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544bab-6721-4e20-988e-52d00c254f01 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:41.158533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2544bab-6721-4e20-988e-52d00c254f01 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:41.160271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 422/838] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:41.165286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84ffdc-177e-434c-b9e9-ee9aca94a512 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:41.165576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84ffdc-177e-434c-b9e9-ee9aca94a512 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:41.165855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84ffdc-177e-434c-b9e9-ee9aca94a512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:41.185046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-af84ffdc-177e-434c-b9e9-ee9aca94a512 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:41.186239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-af84ffdc-177e-434c-b9e9-ee9aca94a512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:41.189207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:41.265814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 417/839] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:10:41.540559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2544bab-6721-4e20-988e-52d00c254f01 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:41.622891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a915bb-eb04-4808-9079-11bca5d6ce46 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:41.623442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a915bb-eb04-4808-9079-11bca5d6ce46 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:41.623442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a915bb-eb04-4808-9079-11bca5d6ce46 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:41.625091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 418/840] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:41.633935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca7e07-56a6-4ac2-94da-a3b0675c7527 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:41.634392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca7e07-56a6-4ac2-94da-a3b0675c7527 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:41.635354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca7e07-56a6-4ac2-94da-a3b0675c7527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:41.654136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2544bab-6721-4e20-988e-52d00c254f01 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:41.655538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0aca7e07-56a6-4ac2-94da-a3b0675c7527 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:41.657067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0aca7e07-56a6-4ac2-94da-a3b0675c7527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:41.671034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:41.706369 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970283e-0f60-4c78-b6d8-3fe92478d424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:41.707013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970283e-0f60-4c78-b6d8-3fe92478d424 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:41.707287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970283e-0f60-4c78-b6d8-3fe92478d424 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:41.707555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970283e-0f60-4c78-b6d8-3fe92478d424 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:41.708148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970283e-0f60-4c78-b6d8-3fe92478d424 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:41.710347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970283e-0f60-4c78-b6d8-3fe92478d424 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7970283e-0f60-4c78-b6d8-3fe92478d424 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:41.724793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:41.725856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:41.726701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 423/841] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:41.733246 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970283e-0f60-4c78-b6d8-3fe92478d424 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:41.756562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 419/842] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:10:41.847591 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970283e-0f60-4c78-b6d8-3fe92478d424 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:41.885354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bb351-6b2f-4e00-9c9d-24b176c98dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:41.886020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bb351-6b2f-4e00-9c9d-24b176c98dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:41.886292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bb351-6b2f-4e00-9c9d-24b176c98dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:41.886564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bb351-6b2f-4e00-9c9d-24b176c98dc2 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:41.887289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bb351-6b2f-4e00-9c9d-24b176c98dc2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:41.887592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bb351-6b2f-4e00-9c9d-24b176c98dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:41.889017 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bb351-6b2f-4e00-9c9d-24b176c98dc2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:41.894110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 424/843] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:41.899790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca7e07-56a6-4ac2-94da-a3b0675c7527 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:41.977586 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:42.011064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca7e07-56a6-4ac2-94da-a3b0675c7527 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:42.046725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5a832-ef0b-452a-ad25-7357241ba5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:42.047693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5a832-ef0b-452a-ad25-7357241ba5df None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.048196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5a832-ef0b-452a-ad25-7357241ba5df None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.048699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5a832-ef0b-452a-ad25-7357241ba5df None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.049536 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5a832-ef0b-452a-ad25-7357241ba5df None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:42.050057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5a832-ef0b-452a-ad25-7357241ba5df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:42.051629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5a832-ef0b-452a-ad25-7357241ba5df None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:42.064534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 420/844] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:42.071324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bb351-6b2f-4e00-9c9d-24b176c98dc2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:42.123748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:42.195231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bb351-6b2f-4e00-9c9d-24b176c98dc2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:42.234770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb58ef-e706-4918-8b95-378398798136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:42.235768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb58ef-e706-4918-8b95-378398798136 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.236251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb58ef-e706-4918-8b95-378398798136 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.236781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb58ef-e706-4918-8b95-378398798136 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.249314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb58ef-e706-4918-8b95-378398798136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d2cb58ef-e706-4918-8b95-378398798136 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:42.254502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:42.255431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:42.256273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 425/845] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:10:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:42.263065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5a832-ef0b-452a-ad25-7357241ba5df None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:42.373701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5a832-ef0b-452a-ad25-7357241ba5df None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:42.409430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cc392-528a-4481-8eab-d65cc482d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:42.409956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cc392-528a-4481-8eab-d65cc482d13f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.410239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cc392-528a-4481-8eab-d65cc482d13f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.410512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cc392-528a-4481-8eab-d65cc482d13f None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.411248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cc392-528a-4481-8eab-d65cc482d13f None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:42.411553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cc392-528a-4481-8eab-d65cc482d13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:42.412971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cc392-528a-4481-8eab-d65cc482d13f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:42.414438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:42.422905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 421/846] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:10:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:42.428207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb58ef-e706-4918-8b95-378398798136 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:42.538146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb58ef-e706-4918-8b95-378398798136 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:42.539643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:42.576668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3755ed-6c5a-4726-9d9d-fb1c402ace96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:42.577306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3755ed-6c5a-4726-9d9d-fb1c402ace96 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.577588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3755ed-6c5a-4726-9d9d-fb1c402ace96 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.577865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3755ed-6c5a-4726-9d9d-fb1c402ace96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.589052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3755ed-6c5a-4726-9d9d-fb1c402ace96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:42.590523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3755ed-6c5a-4726-9d9d-fb1c402ace96 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1b3755ed-6c5a-4726-9d9d-fb1c402ace96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:42.605536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:42.606514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:42.607392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:42.607392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:42.607392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:42.607392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 426/847] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:42.612992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cc392-528a-4481-8eab-d65cc482d13f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:42.625939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c53b4-fc3d-4649-9f7b-c16aa98669b0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.626279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c53b4-fc3d-4649-9f7b-c16aa98669b0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.626560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c53b4-fc3d-4649-9f7b-c16aa98669b0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.629007 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 427/848] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:42.635758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1e63e-539c-4c20-88cf-92a88c354d37 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.636058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1e63e-539c-4c20-88cf-92a88c354d37 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.636315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1e63e-539c-4c20-88cf-92a88c354d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.656691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e7c1e63e-539c-4c20-88cf-92a88c354d37 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:42.658000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7c1e63e-539c-4c20-88cf-92a88c354d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:42.744559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cc392-528a-4481-8eab-d65cc482d13f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:42.760219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 428/849] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:42.783994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe34f5-9f28-4d5d-9bca-b8a5d9ef886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:42.784987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe34f5-9f28-4d5d-9bca-b8a5d9ef886b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.785460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe34f5-9f28-4d5d-9bca-b8a5d9ef886b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.785908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe34f5-9f28-4d5d-9bca-b8a5d9ef886b None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.786733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe34f5-9f28-4d5d-9bca-b8a5d9ef886b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:42.787207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe34f5-9f28-4d5d-9bca-b8a5d9ef886b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:42.788855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe34f5-9f28-4d5d-9bca-b8a5d9ef886b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:42.794893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c7ab3-e039-47bd-8130-0e3419458906 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.795173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c7ab3-e039-47bd-8130-0e3419458906 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.795445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c7ab3-e039-47bd-8130-0e3419458906 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.797036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 429/850] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:10:42.801726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20facf-36ba-4b12-9390-84023dc3214c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.802001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20facf-36ba-4b12-9390-84023dc3214c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.802272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20facf-36ba-4b12-9390-84023dc3214c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.804268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 422/851] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:42.810794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe34f5-9f28-4d5d-9bca-b8a5d9ef886b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:42.820100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ac20facf-36ba-4b12-9390-84023dc3214c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:42.821322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac20facf-36ba-4b12-9390-84023dc3214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:42.882162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:42.899229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 430/852] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:10:42.930097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe34f5-9f28-4d5d-9bca-b8a5d9ef886b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:42.968783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ee571-df23-4f37-9494-ccc5d7d2561e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:42.969757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ee571-df23-4f37-9494-ccc5d7d2561e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:42.970233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ee571-df23-4f37-9494-ccc5d7d2561e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:42.970909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ee571-df23-4f37-9494-ccc5d7d2561e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:42.984888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ee571-df23-4f37-9494-ccc5d7d2561e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:42.986679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ee571-df23-4f37-9494-ccc5d7d2561e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-db7ee571-df23-4f37-9494-ccc5d7d2561e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:42.999473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:43.000485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:43.001457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:43.003741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 423/853] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:10:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:43.009239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20facf-36ba-4b12-9390-84023dc3214c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:43.026514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:43.064756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ee571-df23-4f37-9494-ccc5d7d2561e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:43.128707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20facf-36ba-4b12-9390-84023dc3214c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:43.172899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5755e-c775-444b-a0d4-9f41af44486b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:43.173524 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5755e-c775-444b-a0d4-9f41af44486b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:43.173802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5755e-c775-444b-a0d4-9f41af44486b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:43.174075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5755e-c775-444b-a0d4-9f41af44486b None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:43.174977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5755e-c775-444b-a0d4-9f41af44486b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:43.175282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5755e-c775-444b-a0d4-9f41af44486b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:43.177372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5755e-c775-444b-a0d4-9f41af44486b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:43.183071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ee571-df23-4f37-9494-ccc5d7d2561e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:43.196266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 431/854] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Aug 19 15:10:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:43.197933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5755e-c775-444b-a0d4-9f41af44486b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:43.204821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:43.226493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575730a-706c-4971-92d2-8f9887ecd6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:43.227283 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575730a-706c-4971-92d2-8f9887ecd6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:43.227688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575730a-706c-4971-92d2-8f9887ecd6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:43.227967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575730a-706c-4971-92d2-8f9887ecd6a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:43.241038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575730a-706c-4971-92d2-8f9887ecd6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b575730a-706c-4971-92d2-8f9887ecd6a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:43.246768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:43.247729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 424/855] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:43.253544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575730a-706c-4971-92d2-8f9887ecd6a6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:43.283900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:43.319240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5755e-c775-444b-a0d4-9f41af44486b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:43.357237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec75d2-17e6-464d-8830-a70166d5cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:43.357845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec75d2-17e6-464d-8830-a70166d5cb9a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:43.358121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec75d2-17e6-464d-8830-a70166d5cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:43.358376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec75d2-17e6-464d-8830-a70166d5cb9a None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:43.359068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec75d2-17e6-464d-8830-a70166d5cb9a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:43.360579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec75d2-17e6-464d-8830-a70166d5cb9a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:43.364959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575730a-706c-4971-92d2-8f9887ecd6a6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:43.375404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 432/856] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:10:43 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:43.376206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec75d2-17e6-464d-8830-a70166d5cb9a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:43.419552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06974128-9096-4e6b-8367-cc440e86f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:43.420604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06974128-9096-4e6b-8367-cc440e86f2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:43.421271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06974128-9096-4e6b-8367-cc440e86f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:43.421915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06974128-9096-4e6b-8367-cc440e86f2a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:43.476568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:43.506819 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06974128-9096-4e6b-8367-cc440e86f2a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:43.510489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06974128-9096-4e6b-8367-cc440e86f2a4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:43.518399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec75d2-17e6-464d-8830-a70166d5cb9a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:43.555468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:43.562789 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94cb75-1b44-4f18-a4b2-02d601d74ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:43.563584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94cb75-1b44-4f18-a4b2-02d601d74ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:43.563858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94cb75-1b44-4f18-a4b2-02d601d74ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:43.564521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94cb75-1b44-4f18-a4b2-02d601d74ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:43.566421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94cb75-1b44-4f18-a4b2-02d601d74ab4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:43.566801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94cb75-1b44-4f18-a4b2-02d601d74ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:43.568231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94cb75-1b44-4f18-a4b2-02d601d74ab4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:43.580154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 433/857] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:43.586407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94cb75-1b44-4f18-a4b2-02d601d74ab4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:43.630879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 425/858] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:43 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:10:43.639079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06974128-9096-4e6b-8367-cc440e86f2a4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:43.721739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:43.728377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94cb75-1b44-4f18-a4b2-02d601d74ab4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:43.766782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06974128-9096-4e6b-8367-cc440e86f2a4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:43.769290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87781e18-0a33-464d-abfa-aedda889b2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:43.769928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87781e18-0a33-464d-abfa-aedda889b2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:43.770234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87781e18-0a33-464d-abfa-aedda889b2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:43.770548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87781e18-0a33-464d-abfa-aedda889b2a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:43.782186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87781e18-0a33-464d-abfa-aedda889b2a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:43.783710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87781e18-0a33-464d-abfa-aedda889b2a5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-87781e18-0a33-464d-abfa-aedda889b2a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:43.799432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:43.800434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:43.801343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:43.801343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:43.801343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:43.805541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 434/859] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:43.806353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87781e18-0a33-464d-abfa-aedda889b2a5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:43.810047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80635426-9448-45ea-b9f3-52c27d1a283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:43.811600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80635426-9448-45ea-b9f3-52c27d1a283c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:43.811884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80635426-9448-45ea-b9f3-52c27d1a283c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:43.812157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80635426-9448-45ea-b9f3-52c27d1a283c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:43.826127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80635426-9448-45ea-b9f3-52c27d1a283c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:43.828241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80635426-9448-45ea-b9f3-52c27d1a283c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-80635426-9448-45ea-b9f3-52c27d1a283c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:43.849246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:43.851513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:43.854227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:43.854227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:43.854227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:43.854227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 426/860] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:43.860409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80635426-9448-45ea-b9f3-52c27d1a283c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:43.862139 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:43.927564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87781e18-0a33-464d-abfa-aedda889b2a5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:43.974227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c70f82-c20f-47fa-ba49-5cfbc051dde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:43.975389 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c70f82-c20f-47fa-ba49-5cfbc051dde1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:43.975850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c70f82-c20f-47fa-ba49-5cfbc051dde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:43.976609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c70f82-c20f-47fa-ba49-5cfbc051dde1 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:43.977453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c70f82-c20f-47fa-ba49-5cfbc051dde1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:43.978101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c70f82-c20f-47fa-ba49-5cfbc051dde1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:43.980687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c70f82-c20f-47fa-ba49-5cfbc051dde1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:43.982619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80635426-9448-45ea-b9f3-52c27d1a283c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:43.996202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 435/861] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:44.001612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c70f82-c20f-47fa-ba49-5cfbc051dde1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:44.011676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:44.028636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ddfc6-0256-4655-9743-0536e2032b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:44.029249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ddfc6-0256-4655-9743-0536e2032b91 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.029585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ddfc6-0256-4655-9743-0536e2032b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.029859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ddfc6-0256-4655-9743-0536e2032b91 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.030875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ddfc6-0256-4655-9743-0536e2032b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:44.031176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ddfc6-0256-4655-9743-0536e2032b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:44.032798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ddfc6-0256-4655-9743-0536e2032b91 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:44.052466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 427/862] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:44.055279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ddfc6-0256-4655-9743-0536e2032b91 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:44.124787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c70f82-c20f-47fa-ba49-5cfbc051dde1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:44.133296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:44.168774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dad0a-5a80-4dbb-af9e-99aafffcede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:44.169406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dad0a-5a80-4dbb-af9e-99aafffcede2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.169683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dad0a-5a80-4dbb-af9e-99aafffcede2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.169951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dad0a-5a80-4dbb-af9e-99aafffcede2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.173944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ddfc6-0256-4655-9743-0536e2032b91 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:44.177129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dad0a-5a80-4dbb-af9e-99aafffcede2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:44.178587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dad0a-5a80-4dbb-af9e-99aafffcede2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-574dad0a-5a80-4dbb-af9e-99aafffcede2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:44.190552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:44.191511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:44.192597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 436/863] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:10:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:44.193185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d532a0c-a8b4-46ee-8cb5-2a3f0b95d23d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.193185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d532a0c-a8b4-46ee-8cb5-2a3f0b95d23d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.193816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d532a0c-a8b4-46ee-8cb5-2a3f0b95d23d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.196677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 437/864] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:44.199019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d532a0c-a8b4-46ee-8cb5-2a3f0b95d23d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:44.216706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5267d5e-59cd-4676-99a4-a2b8819c539a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:44.217337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5267d5e-59cd-4676-99a4-a2b8819c539a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.217620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5267d5e-59cd-4676-99a4-a2b8819c539a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.217893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5267d5e-59cd-4676-99a4-a2b8819c539a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.225009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5267d5e-59cd-4676-99a4-a2b8819c539a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:44.226476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5267d5e-59cd-4676-99a4-a2b8819c539a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a5267d5e-59cd-4676-99a4-a2b8819c539a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:44.238641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:44.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 428/865] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646f9c-993d-4283-8450-22f1f8edfbc6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.241617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646f9c-993d-4283-8450-22f1f8edfbc6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.242223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646f9c-993d-4283-8450-22f1f8edfbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.259128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3e646f9c-993d-4283-8450-22f1f8edfbc6 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:44.260996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3e646f9c-993d-4283-8450-22f1f8edfbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:44.302954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:44.320260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d532a0c-a8b4-46ee-8cb5-2a3f0b95d23d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:44.344416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 429/866] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:44 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:44.344416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e646f9c-993d-4283-8450-22f1f8edfbc6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:44.376569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3255c2-28d6-4716-b6b4-0560224f8ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:44.377178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3255c2-28d6-4716-b6b4-0560224f8ffb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.377452 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3255c2-28d6-4716-b6b4-0560224f8ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.377767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3255c2-28d6-4716-b6b4-0560224f8ffb None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.378493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3255c2-28d6-4716-b6b4-0560224f8ffb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:44.378792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3255c2-28d6-4716-b6b4-0560224f8ffb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:44.380298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3255c2-28d6-4716-b6b4-0560224f8ffb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:44.399883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 438/867] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Aug 19 15:10:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:44.405335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3255c2-28d6-4716-b6b4-0560224f8ffb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:44.446413 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:44.458746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e646f9c-993d-4283-8450-22f1f8edfbc6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:44.497891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf87ec-49c4-4d60-8827-19ce93b8b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:44.498419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf87ec-49c4-4d60-8827-19ce93b8b10d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.498720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf87ec-49c4-4d60-8827-19ce93b8b10d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.498969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebf87ec-49c4-4d60-8827-19ce93b8b10d None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.499662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf87ec-49c4-4d60-8827-19ce93b8b10d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:44.499965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf87ec-49c4-4d60-8827-19ce93b8b10d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:44.501645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebf87ec-49c4-4d60-8827-19ce93b8b10d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:44.518834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 430/868] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:44.519926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf87ec-49c4-4d60-8827-19ce93b8b10d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:44.552744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3255c2-28d6-4716-b6b4-0560224f8ffb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:44.562977 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:44.596082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02349b-25b0-460e-abc4-e064a8803d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:44.597168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02349b-25b0-460e-abc4-e064a8803d16 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.597706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02349b-25b0-460e-abc4-e064a8803d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.598213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02349b-25b0-460e-abc4-e064a8803d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.633295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebf87ec-49c4-4d60-8827-19ce93b8b10d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:44.672259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfef0a5-3304-435e-893b-1ee32ba85197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:44.673350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfef0a5-3304-435e-893b-1ee32ba85197 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.673627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfef0a5-3304-435e-893b-1ee32ba85197 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.673894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfef0a5-3304-435e-893b-1ee32ba85197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.674974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfef0a5-3304-435e-893b-1ee32ba85197 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=4d4a71b6005343cda93a92465067a7d7) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:44.675274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfef0a5-3304-435e-893b-1ee32ba85197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:44.676742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfef0a5-3304-435e-893b-1ee32ba85197 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:44.683880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 431/869] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:44 2022] GET /identity/v3/role_assignments?user.id=4d4a71b6005343cda93a92465067a7d7&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:44.684477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02349b-25b0-460e-abc4-e064a8803d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:44.684683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfef0a5-3304-435e-893b-1ee32ba85197 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:44.686284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02349b-25b0-460e-abc4-e064a8803d16 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:44.720267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:44.811667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:44.816902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfef0a5-3304-435e-893b-1ee32ba85197 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:44.820573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 439/870] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:44 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:10:44.823475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02349b-25b0-460e-abc4-e064a8803d16 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:44.859616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d9f39-945f-420b-9782-e5f7e8490b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:44.860656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d9f39-945f-420b-9782-e5f7e8490b38 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.861166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d9f39-945f-420b-9782-e5f7e8490b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.861761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d9f39-945f-420b-9782-e5f7e8490b38 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:44.862730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d9f39-945f-420b-9782-e5f7e8490b38 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:44.864350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d9f39-945f-420b-9782-e5f7e8490b38 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:44.885221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 432/871] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:10:44 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:44.946634 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02349b-25b0-460e-abc4-e064a8803d16 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:44.977186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:44.997455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b4b70-476e-40af-afa6-6945e1739eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:44.998101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b4b70-476e-40af-afa6-6945e1739eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:44.998376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b4b70-476e-40af-afa6-6945e1739eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:44.998642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b4b70-476e-40af-afa6-6945e1739eb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:45.010053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b4b70-476e-40af-afa6-6945e1739eb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:45.011955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b4b70-476e-40af-afa6-6945e1739eb4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-723b4b70-476e-40af-afa6-6945e1739eb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:45.030335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:45.031446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:45.032324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:45.032324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:45.032324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:45.032324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 440/872] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:45.038780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d9f39-945f-420b-9782-e5f7e8490b38 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:45.154631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d9f39-945f-420b-9782-e5f7e8490b38 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:45.189994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48338def-0fa7-4c02-a2d3-8e86709b421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:45.190599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48338def-0fa7-4c02-a2d3-8e86709b421a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:45.190879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48338def-0fa7-4c02-a2d3-8e86709b421a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:45.191152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48338def-0fa7-4c02-a2d3-8e86709b421a None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:45.191797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48338def-0fa7-4c02-a2d3-8e86709b421a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:45.192154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48338def-0fa7-4c02-a2d3-8e86709b421a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:45.193627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48338def-0fa7-4c02-a2d3-8e86709b421a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:45.206286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 433/873] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:45.212214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b4b70-476e-40af-afa6-6945e1739eb4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:45.322117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b4b70-476e-40af-afa6-6945e1739eb4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:45.358373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e40682-ec39-453a-b577-55e65b4833e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:45.359656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e40682-ec39-453a-b577-55e65b4833e1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:45.360188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e40682-ec39-453a-b577-55e65b4833e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:45.360698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e40682-ec39-453a-b577-55e65b4833e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:45.368238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e40682-ec39-453a-b577-55e65b4833e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:45.370023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e40682-ec39-453a-b577-55e65b4833e1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c1e40682-ec39-453a-b577-55e65b4833e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:45.383301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:45.384225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:45.385918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:45.385918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:45.385918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:45.385918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:45.385918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:45.385918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:45.385918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:45.385918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:45.388243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 441/874] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:45.394266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48338def-0fa7-4c02-a2d3-8e86709b421a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:45.415705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:45.471877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a19a4-8f8e-41ae-b6c8-d77e9e5e8b39 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:45.472209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a19a4-8f8e-41ae-b6c8-d77e9e5e8b39 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:45.472466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a19a4-8f8e-41ae-b6c8-d77e9e5e8b39 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:45.474120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 442/875] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:10:45.479294 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a565598-9270-41b2-9668-1aec41bbfb6f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:45.479579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a565598-9270-41b2-9668-1aec41bbfb6f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:45.479858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a565598-9270-41b2-9668-1aec41bbfb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:45.501546 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6a565598-9270-41b2-9668-1aec41bbfb6f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:45.503265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6a565598-9270-41b2-9668-1aec41bbfb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:45.507026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48338def-0fa7-4c02-a2d3-8e86709b421a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:45.535547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:45.547081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb114017-057d-4665-a3d2-de31afc8b3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:45.547950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb114017-057d-4665-a3d2-de31afc8b3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:45.548427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb114017-057d-4665-a3d2-de31afc8b3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:45.548903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb114017-057d-4665-a3d2-de31afc8b3e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:45.549770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb114017-057d-4665-a3d2-de31afc8b3e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:45.550248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb114017-057d-4665-a3d2-de31afc8b3e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:45.551855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb114017-057d-4665-a3d2-de31afc8b3e4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:45.564576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 434/876] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:45.572844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb114017-057d-4665-a3d2-de31afc8b3e4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:45.592479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 443/877] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:10:45.664407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:45.688622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb114017-057d-4665-a3d2-de31afc8b3e4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:45.723556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7db13b-2157-47c0-a0b7-1255d94aa060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:45.724204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7db13b-2157-47c0-a0b7-1255d94aa060 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:45.724546 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7db13b-2157-47c0-a0b7-1255d94aa060 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:45.724818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7db13b-2157-47c0-a0b7-1255d94aa060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:45.725582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7db13b-2157-47c0-a0b7-1255d94aa060 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=383ff491387642a3ab45ab692f94189c) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:45.725887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7db13b-2157-47c0-a0b7-1255d94aa060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:45.727316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7db13b-2157-47c0-a0b7-1255d94aa060 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:45.734329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 435/878] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:45 2022] GET /identity/v3/role_assignments?user.id=383ff491387642a3ab45ab692f94189c&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:45.870909 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a565598-9270-41b2-9668-1aec41bbfb6f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:45.981896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a565598-9270-41b2-9668-1aec41bbfb6f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:46.018898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b36b7b-3425-4612-91c7-98cf05a19ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:46.019532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b36b7b-3425-4612-91c7-98cf05a19ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:46.019806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b36b7b-3425-4612-91c7-98cf05a19ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:46.020075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b36b7b-3425-4612-91c7-98cf05a19ed4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:46.020681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b36b7b-3425-4612-91c7-98cf05a19ed4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:46.022055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b36b7b-3425-4612-91c7-98cf05a19ed4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b3b36b7b-3425-4612-91c7-98cf05a19ed4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:46.032146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:46.033155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:46.034054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 444/879] 10.222.0.75 () {56 vars in 1132 bytes} [Fri Aug 19 15:10:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:46.042537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7db13b-2157-47c0-a0b7-1255d94aa060 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:46.153786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7db13b-2157-47c0-a0b7-1255d94aa060 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:46.188333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915a349-5006-4ace-a401-700dbeb1f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:46.188955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915a349-5006-4ace-a401-700dbeb1f675 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:46.189232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915a349-5006-4ace-a401-700dbeb1f675 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:46.189502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915a349-5006-4ace-a401-700dbeb1f675 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:46.190125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915a349-5006-4ace-a401-700dbeb1f675 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:46.190421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915a349-5006-4ace-a401-700dbeb1f675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:46.191790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915a349-5006-4ace-a401-700dbeb1f675 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:46.196814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 436/880] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:10:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:46.203126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b36b7b-3425-4612-91c7-98cf05a19ed4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:46.237651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:46.317888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b36b7b-3425-4612-91c7-98cf05a19ed4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:46.359760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b0e13-c950-457d-8d07-2b625d18cde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:46.360364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b0e13-c950-457d-8d07-2b625d18cde8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:46.360663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b0e13-c950-457d-8d07-2b625d18cde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:46.360932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b0e13-c950-457d-8d07-2b625d18cde8 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:46.361557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b0e13-c950-457d-8d07-2b625d18cde8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:46.361853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b0e13-c950-457d-8d07-2b625d18cde8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:46.363226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b0e13-c950-457d-8d07-2b625d18cde8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:46.368550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 445/881] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:10:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:46.374789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915a349-5006-4ace-a401-700dbeb1f675 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:46.384757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:46.481737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915a349-5006-4ace-a401-700dbeb1f675 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:46.516780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508b197-4d2d-4402-827a-cfdd46c528b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:46.517442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b197-4d2d-4402-827a-cfdd46c528b5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:46.517796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b197-4d2d-4402-827a-cfdd46c528b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:46.518042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b197-4d2d-4402-827a-cfdd46c528b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:46.518739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508b197-4d2d-4402-827a-cfdd46c528b5 None admin] RBAC: Adding query filter params (interface=public, service_id=770edb4995f848a08697c735ca318ba7, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:46.519042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508b197-4d2d-4402-827a-cfdd46c528b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:46.520421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508b197-4d2d-4402-827a-cfdd46c528b5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:46.529156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 437/882] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Aug 19 15:10:46 2022] GET /identity/v3/endpoints?service_id=770edb4995f848a08697c735ca318ba7&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:46.534689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b0e13-c950-457d-8d07-2b625d18cde8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:46.631808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:46.645079 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b0e13-c950-457d-8d07-2b625d18cde8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:46.686361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3ecee-c95f-4e3a-80a4-13175cd31b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:46.686973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3ecee-c95f-4e3a-80a4-13175cd31b4b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:46.687256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3ecee-c95f-4e3a-80a4-13175cd31b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:46.687532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3ecee-c95f-4e3a-80a4-13175cd31b4b None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:46.688146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3ecee-c95f-4e3a-80a4-13175cd31b4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:46.689592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3ecee-c95f-4e3a-80a4-13175cd31b4b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:46.695939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 446/883] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Aug 19 15:10:46 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:10:46.964711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a6e19-1444-446b-a3e1-2ff7592ff226 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:46.965056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a6e19-1444-446b-a3e1-2ff7592ff226 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:46.965159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a6e19-1444-446b-a3e1-2ff7592ff226 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:46.966746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 438/884] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:46.972024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298a879-727d-4bc2-abde-522ab5f5a4d1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:46.972304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298a879-727d-4bc2-abde-522ab5f5a4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:46.972602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298a879-727d-4bc2-abde-522ab5f5a4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:46.991570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0298a879-727d-4bc2-abde-522ab5f5a4d1 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:46.992830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0298a879-727d-4bc2-abde-522ab5f5a4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:47.010586 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bffa52-780c-4a05-868f-39e6e965602a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.010890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bffa52-780c-4a05-868f-39e6e965602a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.011148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bffa52-780c-4a05-868f-39e6e965602a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.012857 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 439/885] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:47.017636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56ebab-224a-4448-b17c-9854d2a78fd6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.018026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56ebab-224a-4448-b17c-9854d2a78fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.018405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56ebab-224a-4448-b17c-9854d2a78fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.039496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9a56ebab-224a-4448-b17c-9854d2a78fd6 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:47.041009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9a56ebab-224a-4448-b17c-9854d2a78fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:47.087809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 447/886] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:10:47.116333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:47.122365 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 440/887] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:47.364666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298a879-727d-4bc2-abde-522ab5f5a4d1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:47.400201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56ebab-224a-4448-b17c-9854d2a78fd6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:47.474976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:47.497553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298a879-727d-4bc2-abde-522ab5f5a4d1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:47.519068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56ebab-224a-4448-b17c-9854d2a78fd6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:47.534528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443e109-2b32-4ce7-bfcd-4ad018926944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:47.535162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443e109-2b32-4ce7-bfcd-4ad018926944 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.535439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443e109-2b32-4ce7-bfcd-4ad018926944 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.535709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443e109-2b32-4ce7-bfcd-4ad018926944 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.548795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443e109-2b32-4ce7-bfcd-4ad018926944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1443e109-2b32-4ce7-bfcd-4ad018926944 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:47.553439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:47.555394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:47.556737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 448/888] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Aug 19 15:10:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:47.562168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443e109-2b32-4ce7-bfcd-4ad018926944 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:47.566067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23b51f-c4db-474e-8830-d2adeef60abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:47.566958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23b51f-c4db-474e-8830-d2adeef60abb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.567455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23b51f-c4db-474e-8830-d2adeef60abb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.567927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23b51f-c4db-474e-8830-d2adeef60abb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.568821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23b51f-c4db-474e-8830-d2adeef60abb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:47.570426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23b51f-c4db-474e-8830-d2adeef60abb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-2f23b51f-c4db-474e-8830-d2adeef60abb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:47.580321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:47.581390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 15:10:47.582236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:47.584963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 441/889] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:47.591181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23b51f-c4db-474e-8830-d2adeef60abb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:47.674005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:47.700906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443e109-2b32-4ce7-bfcd-4ad018926944 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:47.722615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23b51f-c4db-474e-8830-d2adeef60abb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:47.738602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb282b-3c66-4dd3-bf40-fbfa4a81e37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:47.739637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb282b-3c66-4dd3-bf40-fbfa4a81e37d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.740138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb282b-3c66-4dd3-bf40-fbfa4a81e37d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.740686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb282b-3c66-4dd3-bf40-fbfa4a81e37d None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.741574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb282b-3c66-4dd3-bf40-fbfa4a81e37d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:47.742156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb282b-3c66-4dd3-bf40-fbfa4a81e37d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:47.743936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb282b-3c66-4dd3-bf40-fbfa4a81e37d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:47.757787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 449/890] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Aug 19 15:10:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:47.759494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb282b-3c66-4dd3-bf40-fbfa4a81e37d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:47.763408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634ccef-1611-4439-9117-df08e6ef1357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:47.764014 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634ccef-1611-4439-9117-df08e6ef1357 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.764291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634ccef-1611-4439-9117-df08e6ef1357 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.764638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634ccef-1611-4439-9117-df08e6ef1357 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.766196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634ccef-1611-4439-9117-df08e6ef1357 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:47.766506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634ccef-1611-4439-9117-df08e6ef1357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:47.767873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634ccef-1611-4439-9117-df08e6ef1357 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:47.770993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:47.773521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 442/891] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:47.780155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634ccef-1611-4439-9117-df08e6ef1357 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:47.878492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb282b-3c66-4dd3-bf40-fbfa4a81e37d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:47.897170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634ccef-1611-4439-9117-df08e6ef1357 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:47.913074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dd5c7-c56b-417b-b120-7a017c0323ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:47.913691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dd5c7-c56b-417b-b120-7a017c0323ef None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.913966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dd5c7-c56b-417b-b120-7a017c0323ef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.914234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dd5c7-c56b-417b-b120-7a017c0323ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.924819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dd5c7-c56b-417b-b120-7a017c0323ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:47.925720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:47.926240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dd5c7-c56b-417b-b120-7a017c0323ef None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:47.935414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca269a19-de9f-47a4-b5ee-0ecb126a3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:47.936010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca269a19-de9f-47a4-b5ee-0ecb126a3d00 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.936372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca269a19-de9f-47a4-b5ee-0ecb126a3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.936684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca269a19-de9f-47a4-b5ee-0ecb126a3d00 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.937344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca269a19-de9f-47a4-b5ee-0ecb126a3d00 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:47.937705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca269a19-de9f-47a4-b5ee-0ecb126a3d00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:47.939128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca269a19-de9f-47a4-b5ee-0ecb126a3d00 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6a2dd5c7-c56b-417b-b120-7a017c0323ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:47.941498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:47.942453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:47.943370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:47.943370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:47.943370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:47.943370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 450/892] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:47.944172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f41f54-42d7-45a8-a531-7aa9db6d2700 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.944172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f41f54-42d7-45a8-a531-7aa9db6d2700 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.944527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f41f54-42d7-45a8-a531-7aa9db6d2700 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.945498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 451/893] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:47.949982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f41f54-42d7-45a8-a531-7aa9db6d2700 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:47.951118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 443/894] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:47.953942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69135469-6ba6-44f7-9249-21e168aa888e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:47.954230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69135469-6ba6-44f7-9249-21e168aa888e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:47.954503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69135469-6ba6-44f7-9249-21e168aa888e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:47.972574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-69135469-6ba6-44f7-9249-21e168aa888e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:47.973762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-69135469-6ba6-44f7-9249-21e168aa888e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:48.019247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:48.048449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 444/895] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:48.048449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69135469-6ba6-44f7-9249-21e168aa888e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:48.074518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f41f54-42d7-45a8-a531-7aa9db6d2700 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:48.113941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4556dc-b2d8-4b74-8b98-6e449d95d66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:48.115071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4556dc-b2d8-4b74-8b98-6e449d95d66d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.115857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4556dc-b2d8-4b74-8b98-6e449d95d66d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.116308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4556dc-b2d8-4b74-8b98-6e449d95d66d None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.117160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4556dc-b2d8-4b74-8b98-6e449d95d66d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:48.117603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4556dc-b2d8-4b74-8b98-6e449d95d66d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:48.120718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4556dc-b2d8-4b74-8b98-6e449d95d66d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:48.138300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 452/896] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:48.143484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4556dc-b2d8-4b74-8b98-6e449d95d66d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:48.181272 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69135469-6ba6-44f7-9249-21e168aa888e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:48.213188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:48.236289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26545f11-3421-4019-89d1-c02e45dc7711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:48.236931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26545f11-3421-4019-89d1-c02e45dc7711 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.237210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26545f11-3421-4019-89d1-c02e45dc7711 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.237481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26545f11-3421-4019-89d1-c02e45dc7711 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.238192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26545f11-3421-4019-89d1-c02e45dc7711 None admin] RBAC: Adding query filter params (interface=public, service_id=2432b982f0ec4699b8342e0e9dfb0053, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:48.238498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26545f11-3421-4019-89d1-c02e45dc7711 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:48.239878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26545f11-3421-4019-89d1-c02e45dc7711 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:48.246232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 445/897] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Aug 19 15:10:48 2022] GET /identity/v3/endpoints?service_id=2432b982f0ec4699b8342e0e9dfb0053&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:48.251255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26545f11-3421-4019-89d1-c02e45dc7711 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:48.277026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4556dc-b2d8-4b74-8b98-6e449d95d66d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:48.296753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:48.315161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b85d3-7876-40e3-b784-10c25d2d9d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:48.315963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b85d3-7876-40e3-b784-10c25d2d9d4a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.316349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b85d3-7876-40e3-b784-10c25d2d9d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.316742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b85d3-7876-40e3-b784-10c25d2d9d4a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.325020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b85d3-7876-40e3-b784-10c25d2d9d4a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:48.326589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b85d3-7876-40e3-b784-10c25d2d9d4a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6a0b85d3-7876-40e3-b784-10c25d2d9d4a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:48.339388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:48.345467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 453/898] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:10:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:48.346476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b85d3-7876-40e3-b784-10c25d2d9d4a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:48.363535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26545f11-3421-4019-89d1-c02e45dc7711 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:48.402349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203509b-2701-4598-bcd3-5d1ba00c516e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:48.402946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203509b-2701-4598-bcd3-5d1ba00c516e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.403221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203509b-2701-4598-bcd3-5d1ba00c516e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.403490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203509b-2701-4598-bcd3-5d1ba00c516e None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.404095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203509b-2701-4598-bcd3-5d1ba00c516e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:48.405777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203509b-2701-4598-bcd3-5d1ba00c516e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:48.420423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 446/899] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Aug 19 15:10:48 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:10:48.420423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203509b-2701-4598-bcd3-5d1ba00c516e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:48.461484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b85d3-7876-40e3-b784-10c25d2d9d4a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:48.487578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:48.505306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03669c13-399f-4283-97ed-fa4f76248514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:48.505955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03669c13-399f-4283-97ed-fa4f76248514 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.506232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03669c13-399f-4283-97ed-fa4f76248514 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.506502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03669c13-399f-4283-97ed-fa4f76248514 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.507094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03669c13-399f-4283-97ed-fa4f76248514 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:48.508539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03669c13-399f-4283-97ed-fa4f76248514 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-03669c13-399f-4283-97ed-fa4f76248514 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:48.518513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:48.525155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:48.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 454/900] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:10:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:48.528537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03669c13-399f-4283-97ed-fa4f76248514 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:48.561029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203509b-2701-4598-bcd3-5d1ba00c516e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:48.588263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:48.605033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afd818-bf50-49c7-999a-8bcb1c70fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:48.606082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afd818-bf50-49c7-999a-8bcb1c70fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.606082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afd818-bf50-49c7-999a-8bcb1c70fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.606334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afd818-bf50-49c7-999a-8bcb1c70fdd1 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.607198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afd818-bf50-49c7-999a-8bcb1c70fdd1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:48.607553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afd818-bf50-49c7-999a-8bcb1c70fdd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:48.610134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afd818-bf50-49c7-999a-8bcb1c70fdd1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:48.632050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 447/901] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Aug 19 15:10:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:48.638834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afd818-bf50-49c7-999a-8bcb1c70fdd1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:48.661503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03669c13-399f-4283-97ed-fa4f76248514 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:48.710981 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b77c27-8bd3-421d-86cc-86f8f198ee0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:48.711642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b77c27-8bd3-421d-86cc-86f8f198ee0a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.711868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b77c27-8bd3-421d-86cc-86f8f198ee0a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.712132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b77c27-8bd3-421d-86cc-86f8f198ee0a None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.713343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b77c27-8bd3-421d-86cc-86f8f198ee0a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:48.713671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b77c27-8bd3-421d-86cc-86f8f198ee0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:48.715095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b77c27-8bd3-421d-86cc-86f8f198ee0a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:48.720131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 455/902] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:10:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:48.721334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d51c0-af15-4934-bf31-3d05a0903898 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.721623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d51c0-af15-4934-bf31-3d05a0903898 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.721894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d51c0-af15-4934-bf31-3d05a0903898 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.723223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 456/903] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:48.727964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d51c0-af15-4934-bf31-3d05a0903898 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:48.764608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afd818-bf50-49c7-999a-8bcb1c70fdd1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:48.778107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:48.806140 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cefc-a8c5-41ee-b718-a036a17fb802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:48.806990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cefc-a8c5-41ee-b718-a036a17fb802 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.807261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cefc-a8c5-41ee-b718-a036a17fb802 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.807535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cefc-a8c5-41ee-b718-a036a17fb802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.843266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d51c0-af15-4934-bf31-3d05a0903898 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:48.851107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:48.881374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523cefc-a8c5-41ee-b718-a036a17fb802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:48.882378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e97d4e-ca53-42af-beba-ce6f40c3c3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:48.882825 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523cefc-a8c5-41ee-b718-a036a17fb802 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:48.882998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e97d4e-ca53-42af-beba-ce6f40c3c3ba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.883253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e97d4e-ca53-42af-beba-ce6f40c3c3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.883578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e97d4e-ca53-42af-beba-ce6f40c3c3ba None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.884212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e97d4e-ca53-42af-beba-ce6f40c3c3ba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:48.884576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e97d4e-ca53-42af-beba-ce6f40c3c3ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:48.886385 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e97d4e-ca53-42af-beba-ce6f40c3c3ba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:48.890958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 457/904] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:10:48 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:48.898226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb414bb4-e122-4c48-be12-4d6f66a625e3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:48.899462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb414bb4-e122-4c48-be12-4d6f66a625e3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:48.900101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb414bb4-e122-4c48-be12-4d6f66a625e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:48.928419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fb414bb4-e122-4c48-be12-4d6f66a625e3 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:48.930078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fb414bb4-e122-4c48-be12-4d6f66a625e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:49.012596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 448/905] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:48 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:10:49.015871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cefc-a8c5-41ee-b718-a036a17fb802 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:49.028125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 458/906] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:49.045138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:49.129755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cefc-a8c5-41ee-b718-a036a17fb802 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:49.171304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cf9b0-c0dc-43fc-985f-46d7ccc63747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:49.172294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cf9b0-c0dc-43fc-985f-46d7ccc63747 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:49.172839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cf9b0-c0dc-43fc-985f-46d7ccc63747 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:49.173312 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cf9b0-c0dc-43fc-985f-46d7ccc63747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:49.184859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cf9b0-c0dc-43fc-985f-46d7ccc63747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:49.186998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16cf9b0-c0dc-43fc-985f-46d7ccc63747 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a16cf9b0-c0dc-43fc-985f-46d7ccc63747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:49.202113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:49.203062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:49.203956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:49.203956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:49.203956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:49.206064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 449/907] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:49.216423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb414bb4-e122-4c48-be12-4d6f66a625e3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:49.319751 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:49.323543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb414bb4-e122-4c48-be12-4d6f66a625e3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:49.326517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cf9b0-c0dc-43fc-985f-46d7ccc63747 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:49.377328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6dfa9a-985b-492a-9ed4-d2ef3ff1c549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:49.378433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dfa9a-985b-492a-9ed4-d2ef3ff1c549 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:49.378925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dfa9a-985b-492a-9ed4-d2ef3ff1c549 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:49.379406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dfa9a-985b-492a-9ed4-d2ef3ff1c549 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:49.380763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dfa9a-985b-492a-9ed4-d2ef3ff1c549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:49.381289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dfa9a-985b-492a-9ed4-d2ef3ff1c549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:49.383261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6dfa9a-985b-492a-9ed4-d2ef3ff1c549 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:49.399858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 459/908] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:49.405771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:49.408197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6dfa9a-985b-492a-9ed4-d2ef3ff1c549 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:49.457012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cf9b0-c0dc-43fc-985f-46d7ccc63747 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:49.496359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a982f-161d-4283-beb5-a255fa323cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:49.497397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a982f-161d-4283-beb5-a255fa323cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:49.497891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a982f-161d-4283-beb5-a255fa323cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:49.498384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a982f-161d-4283-beb5-a255fa323cd0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:49.499192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a982f-161d-4283-beb5-a255fa323cd0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:49.500818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a982f-161d-4283-beb5-a255fa323cd0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-416a982f-161d-4283-beb5-a255fa323cd0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:49.512222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:49.513343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 15:10:49.514322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:49.517364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 450/909] 10.222.0.75 () {56 vars in 1132 bytes} [Fri Aug 19 15:10:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:49.518761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6dfa9a-985b-492a-9ed4-d2ef3ff1c549 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:49.522699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a982f-161d-4283-beb5-a255fa323cd0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:49.567577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13c321-e280-4305-a45b-0edf9802a5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:49.568219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13c321-e280-4305-a45b-0edf9802a5ec None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:49.568766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13c321-e280-4305-a45b-0edf9802a5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:49.569043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13c321-e280-4305-a45b-0edf9802a5ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:49.578643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13c321-e280-4305-a45b-0edf9802a5ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:49.580031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13c321-e280-4305-a45b-0edf9802a5ec None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0c13c321-e280-4305-a45b-0edf9802a5ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:49.597026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:49.597983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:49.598936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:49.598936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:49.598936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:49.598936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:49.598936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:49.598936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:49.598936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:49.598936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:49.601433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 460/910] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:49.612457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13c321-e280-4305-a45b-0edf9802a5ec None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:49.633081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:49.661264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a982f-161d-4283-beb5-a255fa323cd0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:49.692509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:49.707707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b1f5-146f-4ad0-b33e-4ef3a92cd4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:49.709395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b1f5-146f-4ad0-b33e-4ef3a92cd4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:49.709837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b1f5-146f-4ad0-b33e-4ef3a92cd4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:49.710252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6b1f5-146f-4ad0-b33e-4ef3a92cd4b6 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:49.711033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b1f5-146f-4ad0-b33e-4ef3a92cd4b6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:49.711499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b1f5-146f-4ad0-b33e-4ef3a92cd4b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:49.714311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6b1f5-146f-4ad0-b33e-4ef3a92cd4b6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:49.720847 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 451/911] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:10:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:49.726123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b1f5-146f-4ad0-b33e-4ef3a92cd4b6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13c321-e280-4305-a45b-0edf9802a5ec None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:49.780950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750b591-7c2a-4ba6-8757-bfb698d3bc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:49.781350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750b591-7c2a-4ba6-8757-bfb698d3bc6a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:49.781626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750b591-7c2a-4ba6-8757-bfb698d3bc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:49.781899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750b591-7c2a-4ba6-8757-bfb698d3bc6a None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:49.782592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750b591-7c2a-4ba6-8757-bfb698d3bc6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:49.782989 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750b591-7c2a-4ba6-8757-bfb698d3bc6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:49.784517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750b591-7c2a-4ba6-8757-bfb698d3bc6a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:49.800691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 461/912] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:49.803192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750b591-7c2a-4ba6-8757-bfb698d3bc6a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:49.865853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6b1f5-146f-4ad0-b33e-4ef3a92cd4b6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:49.893985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:49.915570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf4703-2676-4904-8629-8dbc6824a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:49.917067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750b591-7c2a-4ba6-8757-bfb698d3bc6a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:49.917449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf4703-2676-4904-8629-8dbc6824a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:49.917878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf4703-2676-4904-8629-8dbc6824a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:49.918310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf4703-2676-4904-8629-8dbc6824a2e5 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:49.920234 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf4703-2676-4904-8629-8dbc6824a2e5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:49.920722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf4703-2676-4904-8629-8dbc6824a2e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:49.922266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf4703-2676-4904-8629-8dbc6824a2e5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:49.929807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 452/913] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:10:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:49.935889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf4703-2676-4904-8629-8dbc6824a2e5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:49.963417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09325178-338c-42d6-a259-83c4c728e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:49.964833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09325178-338c-42d6-a259-83c4c728e1fc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:49.965515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09325178-338c-42d6-a259-83c4c728e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:49.965928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09325178-338c-42d6-a259-83c4c728e1fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:49.967071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09325178-338c-42d6-a259-83c4c728e1fc None admin] RBAC: Adding query filter params (role.id=68bd116b233640b7b7e07ef743a8be88, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=4d4a71b6005343cda93a92465067a7d7) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:49.967910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09325178-338c-42d6-a259-83c4c728e1fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:49.970040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09325178-338c-42d6-a259-83c4c728e1fc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:49.979103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 462/914] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:49 2022] GET /identity/v3/role_assignments?user.id=4d4a71b6005343cda93a92465067a7d7&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=68bd116b233640b7b7e07ef743a8be88 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:50.013605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:50.074264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf4703-2676-4904-8629-8dbc6824a2e5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:50.112842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee816530-54af-4eff-8654-c171c60e040a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:50.113465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee816530-54af-4eff-8654-c171c60e040a None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:50.113804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee816530-54af-4eff-8654-c171c60e040a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:50.114135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee816530-54af-4eff-8654-c171c60e040a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:50.114868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee816530-54af-4eff-8654-c171c60e040a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:50.116323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee816530-54af-4eff-8654-c171c60e040a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:50.142616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 453/915] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:10:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:50.510465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1f688-aaee-4440-8d11-6a6864727170 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:50.510999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1f688-aaee-4440-8d11-6a6864727170 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:50.511389 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c1f688-aaee-4440-8d11-6a6864727170 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:50.513182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 463/916] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:50.518709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f8dd4-6126-429a-bde2-5124903dd660 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:50.519183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f8dd4-6126-429a-bde2-5124903dd660 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:50.519452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f8dd4-6126-429a-bde2-5124903dd660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:50.538399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-787f8dd4-6126-429a-bde2-5124903dd660 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:50.539654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-787f8dd4-6126-429a-bde2-5124903dd660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:50.616748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 454/917] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:50.900120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1f688-aaee-4440-8d11-6a6864727170 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:50.907868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ed68f-25aa-438d-b922-4fb2e295fc75 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:50.908154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ed68f-25aa-438d-b922-4fb2e295fc75 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:50.908449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ed68f-25aa-438d-b922-4fb2e295fc75 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:50.910063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 455/918] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:10:50.914882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a3dc5-a2cd-4ac8-a1f5-c73fae34b9b5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:50.915166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a3dc5-a2cd-4ac8-a1f5-c73fae34b9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:50.915441 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a3dc5-a2cd-4ac8-a1f5-c73fae34b9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:50.936139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1a5a3dc5-a2cd-4ac8-a1f5-c73fae34b9b5 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:50.937337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1a5a3dc5-a2cd-4ac8-a1f5-c73fae34b9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:50.968250 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:51.019913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c1f688-aaee-4440-8d11-6a6864727170 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:51.024472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 456/919] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:10:51.064439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ce046-553c-42e1-9411-1b34d4780657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:51.065439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ce046-553c-42e1-9411-1b34d4780657 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:51.065927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ce046-553c-42e1-9411-1b34d4780657 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:51.066412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ce046-553c-42e1-9411-1b34d4780657 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:51.067277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ce046-553c-42e1-9411-1b34d4780657 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:51.068885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ce046-553c-42e1-9411-1b34d4780657 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7d7ce046-553c-42e1-9411-1b34d4780657 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:51.078115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:51.079145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 15:10:51.080000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:51.082707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 464/920] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:51.089015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a3dc5-a2cd-4ac8-a1f5-c73fae34b9b5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:51.196940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a3dc5-a2cd-4ac8-a1f5-c73fae34b9b5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:51.233869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491448-46b4-4e62-bd78-0bf69a3b6d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:51.234523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491448-46b4-4e62-bd78-0bf69a3b6d73 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:51.234807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491448-46b4-4e62-bd78-0bf69a3b6d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:51.235075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491448-46b4-4e62-bd78-0bf69a3b6d73 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:51.235702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6491448-46b4-4e62-bd78-0bf69a3b6d73 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:51.236004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6491448-46b4-4e62-bd78-0bf69a3b6d73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:51.237510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6491448-46b4-4e62-bd78-0bf69a3b6d73 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:51.242766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 457/921] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:51.248768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ce046-553c-42e1-9411-1b34d4780657 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:51.262001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:51.295831 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491448-46b4-4e62-bd78-0bf69a3b6d73 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:51.326953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:51.372441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ce046-553c-42e1-9411-1b34d4780657 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:51.406142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491448-46b4-4e62-bd78-0bf69a3b6d73 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:51.414849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334982b2-c75b-4329-8265-c6d678aea5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:51.415463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334982b2-c75b-4329-8265-c6d678aea5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:51.415745 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334982b2-c75b-4329-8265-c6d678aea5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:51.416053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334982b2-c75b-4329-8265-c6d678aea5e1 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:51.416744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334982b2-c75b-4329-8265-c6d678aea5e1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:51.417048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334982b2-c75b-4329-8265-c6d678aea5e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:51.418441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334982b2-c75b-4329-8265-c6d678aea5e1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:51.428675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 465/922] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Aug 19 15:10:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:51.429497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334982b2-c75b-4329-8265-c6d678aea5e1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:51.450309 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53c250-889e-49b7-98c7-e299973315c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:51.450845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53c250-889e-49b7-98c7-e299973315c2 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:51.451123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53c250-889e-49b7-98c7-e299973315c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:51.451395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53c250-889e-49b7-98c7-e299973315c2 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:51.452118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53c250-889e-49b7-98c7-e299973315c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:51.454361 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e53c250-889e-49b7-98c7-e299973315c2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:51.477386 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 458/923] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:10:51 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:10:51.513393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:51.550653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334982b2-c75b-4329-8265-c6d678aea5e1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:51.595548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81abc24-b0bc-4568-b09a-c89cf9848cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:51.596167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81abc24-b0bc-4568-b09a-c89cf9848cd0 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:51.596462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81abc24-b0bc-4568-b09a-c89cf9848cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:51.596736 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81abc24-b0bc-4568-b09a-c89cf9848cd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:51.597402 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81abc24-b0bc-4568-b09a-c89cf9848cd0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:51.598792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81abc24-b0bc-4568-b09a-c89cf9848cd0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:51.622220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 466/924] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:10:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:52.045406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3fb79-69a4-445e-9957-55fc0aa5f84a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:52.046078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3fb79-69a4-445e-9957-55fc0aa5f84a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:52.046078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3fb79-69a4-445e-9957-55fc0aa5f84a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:52.047561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 459/925] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:52.053218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656fefe-0740-4578-877a-2622ada3bacb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:52.054113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656fefe-0740-4578-877a-2622ada3bacb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:52.054610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656fefe-0740-4578-877a-2622ada3bacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:52.076828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d656fefe-0740-4578-877a-2622ada3bacb None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:52.078383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d656fefe-0740-4578-877a-2622ada3bacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:52.117792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c37f6-10b5-438e-8b41-8733c30ce3eb None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:52.118131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c37f6-10b5-438e-8b41-8733c30ce3eb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:52.118413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c37f6-10b5-438e-8b41-8733c30ce3eb None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:52.120843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 460/926] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:52.126069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611b951-9576-487c-9133-fa642ba1a140 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:52.126992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611b951-9576-487c-9133-fa642ba1a140 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:52.127485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611b951-9576-487c-9133-fa642ba1a140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:52.150794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b611b951-9576-487c-9133-fa642ba1a140 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:52.152636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b611b951-9576-487c-9133-fa642ba1a140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:52.164423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 467/927] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:10:52.232208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 461/928] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:52.431307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656fefe-0740-4578-877a-2622ada3bacb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:52.482885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:52.507185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611b951-9576-487c-9133-fa642ba1a140 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:52.527475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:52.538407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656fefe-0740-4578-877a-2622ada3bacb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:52.575232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93158ac9-0339-4c03-886c-757b766a7fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:52.576272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93158ac9-0339-4c03-886c-757b766a7fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:52.576777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93158ac9-0339-4c03-886c-757b766a7fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:52.577265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93158ac9-0339-4c03-886c-757b766a7fd8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:52.587809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93158ac9-0339-4c03-886c-757b766a7fd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-93158ac9-0339-4c03-886c-757b766a7fd8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:52.592808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:52.593742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:10:52.594594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:52.601867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 468/929] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Aug 19 15:10:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:52.603273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93158ac9-0339-4c03-886c-757b766a7fd8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:52.619681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611b951-9576-487c-9133-fa642ba1a140 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:52.656235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8295e-c8a4-4e30-9dfe-ece6f09c7e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:52.656903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8295e-c8a4-4e30-9dfe-ece6f09c7e28 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:52.657194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8295e-c8a4-4e30-9dfe-ece6f09c7e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:52.657456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8295e-c8a4-4e30-9dfe-ece6f09c7e28 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:52.658034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8295e-c8a4-4e30-9dfe-ece6f09c7e28 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:52.659416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8295e-c8a4-4e30-9dfe-ece6f09c7e28 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-93c8295e-c8a4-4e30-9dfe-ece6f09c7e28 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:52.668590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:52.669624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 15:10:52.670484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:52.674777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 462/930] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:52.675701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8295e-c8a4-4e30-9dfe-ece6f09c7e28 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:52.729785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93158ac9-0339-4c03-886c-757b766a7fd8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:52.745607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:52.774460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd5768-e6ae-4c90-98f6-2662145a51fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:52.775117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd5768-e6ae-4c90-98f6-2662145a51fe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:52.775445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd5768-e6ae-4c90-98f6-2662145a51fe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:52.775916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd5768-e6ae-4c90-98f6-2662145a51fe None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:52.777064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd5768-e6ae-4c90-98f6-2662145a51fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:52.777372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd5768-e6ae-4c90-98f6-2662145a51fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:52.778761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd5768-e6ae-4c90-98f6-2662145a51fe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:52.782155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:52.789251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 469/931] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Aug 19 15:10:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:52.796095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd5768-e6ae-4c90-98f6-2662145a51fe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:52.809086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8295e-c8a4-4e30-9dfe-ece6f09c7e28 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:52.847944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc09559-4d3a-4b83-972c-9845d6a19f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:52.849015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc09559-4d3a-4b83-972c-9845d6a19f37 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:52.849604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc09559-4d3a-4b83-972c-9845d6a19f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:52.850038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc09559-4d3a-4b83-972c-9845d6a19f37 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:52.850769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc09559-4d3a-4b83-972c-9845d6a19f37 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:52.851168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc09559-4d3a-4b83-972c-9845d6a19f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:52.852854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc09559-4d3a-4b83-972c-9845d6a19f37 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:52.864077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 463/932] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:52.865093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc09559-4d3a-4b83-972c-9845d6a19f37 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:52.923570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd5768-e6ae-4c90-98f6-2662145a51fe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:52.967461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64148fa5-5853-4b05-8f04-99dc4881ad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:52.968100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64148fa5-5853-4b05-8f04-99dc4881ad9d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:52.968641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64148fa5-5853-4b05-8f04-99dc4881ad9d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:52.968940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64148fa5-5853-4b05-8f04-99dc4881ad9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:52.986155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64148fa5-5853-4b05-8f04-99dc4881ad9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:52.987540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64148fa5-5853-4b05-8f04-99dc4881ad9d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:52.987833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc09559-4d3a-4b83-972c-9845d6a19f37 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-64148fa5-5853-4b05-8f04-99dc4881ad9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:53.003852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:53.005839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:53.007167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:53.007167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:53.007167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:53.007167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 470/933] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:53.013553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64148fa5-5853-4b05-8f04-99dc4881ad9d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:53.031616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017fe31-2706-4353-97b4-9a197b036c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:53.032519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017fe31-2706-4353-97b4-9a197b036c72 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.032802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017fe31-2706-4353-97b4-9a197b036c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.033078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017fe31-2706-4353-97b4-9a197b036c72 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.034210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017fe31-2706-4353-97b4-9a197b036c72 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:53.034515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017fe31-2706-4353-97b4-9a197b036c72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:53.036466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017fe31-2706-4353-97b4-9a197b036c72 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:53.043313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 464/934] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:53.051930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:53.139794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64148fa5-5853-4b05-8f04-99dc4881ad9d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:53.178797 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42dc9d-f488-4714-9506-2c6ffb8bbf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:53.179436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42dc9d-f488-4714-9506-2c6ffb8bbf6a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.179715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42dc9d-f488-4714-9506-2c6ffb8bbf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.180043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42dc9d-f488-4714-9506-2c6ffb8bbf6a None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.180737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42dc9d-f488-4714-9506-2c6ffb8bbf6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:53.181040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42dc9d-f488-4714-9506-2c6ffb8bbf6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:53.182558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42dc9d-f488-4714-9506-2c6ffb8bbf6a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:53.196134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 471/935] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:53.203204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017fe31-2706-4353-97b4-9a197b036c72 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:53.217000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:53.312692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017fe31-2706-4353-97b4-9a197b036c72 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:53.348808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b1482-90ec-4758-985b-c472dec52ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:53.349428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b1482-90ec-4758-985b-c472dec52ded None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.349704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b1482-90ec-4758-985b-c472dec52ded None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.349972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b1482-90ec-4758-985b-c472dec52ded None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.358691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b1482-90ec-4758-985b-c472dec52ded None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:53.360331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b1482-90ec-4758-985b-c472dec52ded None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-143b1482-90ec-4758-985b-c472dec52ded None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:53.373054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:53.375353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:53.377653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 465/936] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:10:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:53.384412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42dc9d-f488-4714-9506-2c6ffb8bbf6a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:53.474493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:53.492092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42dc9d-f488-4714-9506-2c6ffb8bbf6a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:53.514965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245b125-867d-424b-8c36-3ab1604eaa82 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.515289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245b125-867d-424b-8c36-3ab1604eaa82 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.515514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245b125-867d-424b-8c36-3ab1604eaa82 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.517156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 466/937] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:53.525797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb885b-eb26-4e5d-9025-94a1044fa768 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.525797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb885b-eb26-4e5d-9025-94a1044fa768 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.525797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb885b-eb26-4e5d-9025-94a1044fa768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.530495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd86dbf-5525-4681-8ab1-7c82a7e960f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:53.531116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd86dbf-5525-4681-8ab1-7c82a7e960f5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.531395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd86dbf-5525-4681-8ab1-7c82a7e960f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.531667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd86dbf-5525-4681-8ab1-7c82a7e960f5 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.532418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd86dbf-5525-4681-8ab1-7c82a7e960f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:53.532731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd86dbf-5525-4681-8ab1-7c82a7e960f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:53.534241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd86dbf-5525-4681-8ab1-7c82a7e960f5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:53.543514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a5eb885b-eb26-4e5d-9025-94a1044fa768 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:53.545031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a5eb885b-eb26-4e5d-9025-94a1044fa768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:53.553338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 472/938] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Aug 19 15:10:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:53.559248 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd86dbf-5525-4681-8ab1-7c82a7e960f5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:53.629510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:53.636118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 467/939] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:10:53.637489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18c7f-a3d0-437d-be7b-2288a8e19838 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.637770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18c7f-a3d0-437d-be7b-2288a8e19838 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.638043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18c7f-a3d0-437d-be7b-2288a8e19838 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.639297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 468/940] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:53.646134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442859-1441-41ae-95d5-45425fe71099 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.646425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442859-1441-41ae-95d5-45425fe71099 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.646702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71442859-1441-41ae-95d5-45425fe71099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.672877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd86dbf-5525-4681-8ab1-7c82a7e960f5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:53.673826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-71442859-1441-41ae-95d5-45425fe71099 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:53.676515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-71442859-1441-41ae-95d5-45425fe71099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:53.713766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c5fad-62d6-4641-a3ab-eef1c6d5f7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:53.714876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c5fad-62d6-4641-a3ab-eef1c6d5f7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.715354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c5fad-62d6-4641-a3ab-eef1c6d5f7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.715850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c5fad-62d6-4641-a3ab-eef1c6d5f7a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.727912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c5fad-62d6-4641-a3ab-eef1c6d5f7a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:53.729726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c5fad-62d6-4641-a3ab-eef1c6d5f7a5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:53.731975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-655c5fad-62d6-4641-a3ab-eef1c6d5f7a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:53.747839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:53.748881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:53.749817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:53.749817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:53.749817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:53.756280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 473/941] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:53.757702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c5fad-62d6-4641-a3ab-eef1c6d5f7a5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:53.768576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 469/942] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:53.873383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c5fad-62d6-4641-a3ab-eef1c6d5f7a5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:53.909103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a4368-8287-4626-be6a-83452dc147f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:53.909629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4368-8287-4626-be6a-83452dc147f2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:53.909906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4368-8287-4626-be6a-83452dc147f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:53.910178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4368-8287-4626-be6a-83452dc147f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:53.910820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a4368-8287-4626-be6a-83452dc147f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:53.911118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a4368-8287-4626-be6a-83452dc147f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:53.912505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a4368-8287-4626-be6a-83452dc147f2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:53.919884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442859-1441-41ae-95d5-45425fe71099 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:53.930740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 474/943] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:53.935023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a4368-8287-4626-be6a-83452dc147f2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:54.015437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:54.036186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71442859-1441-41ae-95d5-45425fe71099 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.067716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:54.075707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec83ed-f0e6-47f4-96b5-0b7fb6381e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.076342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec83ed-f0e6-47f4-96b5-0b7fb6381e3d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.076661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec83ed-f0e6-47f4-96b5-0b7fb6381e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.076975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec83ed-f0e6-47f4-96b5-0b7fb6381e3d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.077590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec83ed-f0e6-47f4-96b5-0b7fb6381e3d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.077703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a4368-8287-4626-be6a-83452dc147f2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.078982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec83ed-f0e6-47f4-96b5-0b7fb6381e3d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-10ec83ed-f0e6-47f4-96b5-0b7fb6381e3d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:54.088604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:54.089665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.090531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 470/944] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:10:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:54.091330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec83ed-f0e6-47f4-96b5-0b7fb6381e3d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:54.121567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac630fc-cdee-47a3-91b4-78ee72c119f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.122169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac630fc-cdee-47a3-91b4-78ee72c119f0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.122441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac630fc-cdee-47a3-91b4-78ee72c119f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.122709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac630fc-cdee-47a3-91b4-78ee72c119f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.129636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac630fc-cdee-47a3-91b4-78ee72c119f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.131086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac630fc-cdee-47a3-91b4-78ee72c119f0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1ac630fc-cdee-47a3-91b4-78ee72c119f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:54.143159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:54.144652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 475/945] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:54.145564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac630fc-cdee-47a3-91b4-78ee72c119f0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:54.203171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec83ed-f0e6-47f4-96b5-0b7fb6381e3d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.238719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.239524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.239715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.239921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.240605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.242039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.245345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WARNING keystone.server.flask.common [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 19 15:10:54.263472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac630fc-cdee-47a3-91b4-78ee72c119f0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.264420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.sql.core [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 19 15:10:54.264420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 19 15:10:54.264420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [parameters: {'id': 'fd68d2a0c9b444d4a7e2cf5db0c82e03', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 19 15:10:54.264420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81504) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 19 15:10:54.278630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fd68d2a0c9b444d4a7e2cf5db0c82e03.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fd68d2a0c9b444d4a7e2cf5db0c82e03. Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 15:10:54.280723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 19 15:10:54.281866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application yield resource Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.rollback() Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.force_reraise() Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise self.value Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application session.commit() Aug 19 15:10:54.282836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._prepare_impl() Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.flush() Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._flush(objects) Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application compat.raise_( Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application flush_context.execute() Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 19 15:10:54.283739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rec.execute(self) Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 19 15:10:54.284712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 15:10:54.285517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [parameters: {'id': 'fd68d2a0c9b444d4a7e2cf5db0c82e03', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:54.286400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:54.288888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fd68d2a0c9b444d4a7e2cf5db0c82e03. Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 471/946] 10.222.0.75 () {60 vars in 1140 bytes} [Fri Aug 19 15:10:54 2022] POST /identity/v3/projects => generated 262 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:54.290007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:54.312316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6a0d0-b506-4e3f-bdaf-e6ba1df86a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.313151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6a0d0-b506-4e3f-bdaf-e6ba1df86a2e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.313498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6a0d0-b506-4e3f-bdaf-e6ba1df86a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.313837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6a0d0-b506-4e3f-bdaf-e6ba1df86a2e None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.314583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6a0d0-b506-4e3f-bdaf-e6ba1df86a2e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:54.314953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6a0d0-b506-4e3f-bdaf-e6ba1df86a2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.316704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6a0d0-b506-4e3f-bdaf-e6ba1df86a2e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.322244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 476/947] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:10:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:54.323204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6a0d0-b506-4e3f-bdaf-e6ba1df86a2e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:54.345853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:54.406318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1e77f-0303-488c-9d0d-bb18c3038ec0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.440020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6a0d0-b506-4e3f-bdaf-e6ba1df86a2e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.444322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011d647-d2c2-468b-8aa6-0eb8b416ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.444965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011d647-d2c2-468b-8aa6-0eb8b416ef16 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.445247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011d647-d2c2-468b-8aa6-0eb8b416ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.445521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011d647-d2c2-468b-8aa6-0eb8b416ef16 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.446218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011d647-d2c2-468b-8aa6-0eb8b416ef16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:54.446528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011d647-d2c2-468b-8aa6-0eb8b416ef16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.449487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011d647-d2c2-468b-8aa6-0eb8b416ef16 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.469704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 472/948] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:54.469704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011d647-d2c2-468b-8aa6-0eb8b416ef16 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:54.480793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e127571-c708-4448-9da3-dbbf48f0d069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.481506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e127571-c708-4448-9da3-dbbf48f0d069 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.481833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e127571-c708-4448-9da3-dbbf48f0d069 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.482154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e127571-c708-4448-9da3-dbbf48f0d069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.489754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e127571-c708-4448-9da3-dbbf48f0d069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.491205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e127571-c708-4448-9da3-dbbf48f0d069 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4e127571-c708-4448-9da3-dbbf48f0d069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:54.503872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:54.504830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:54.505739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:54.505739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:54.505739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:54.505739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:54.505739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:54.505739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:54.505739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:54.505739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:54.505739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 477/949] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:54.506268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e127571-c708-4448-9da3-dbbf48f0d069 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:54.550506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:54.579607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011d647-d2c2-468b-8aa6-0eb8b416ef16 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.592073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:54.616704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e127571-c708-4448-9da3-dbbf48f0d069 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.623262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17379db-f290-4e92-b167-a11f4964ede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.624056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17379db-f290-4e92-b167-a11f4964ede0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.624930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17379db-f290-4e92-b167-a11f4964ede0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.625359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17379db-f290-4e92-b167-a11f4964ede0 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.626174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17379db-f290-4e92-b167-a11f4964ede0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:54.626994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17379db-f290-4e92-b167-a11f4964ede0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.628892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17379db-f290-4e92-b167-a11f4964ede0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.635074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 473/950] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:10:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:54.635757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17379db-f290-4e92-b167-a11f4964ede0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:54.658423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca20d70-e29a-42ea-b8e7-03aa0572918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.659088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca20d70-e29a-42ea-b8e7-03aa0572918d None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.659372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca20d70-e29a-42ea-b8e7-03aa0572918d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.659644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca20d70-e29a-42ea-b8e7-03aa0572918d None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/4d4a71b6005343cda93a92465067a7d7/roles/68bd116b233640b7b7e07ef743a8be88` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.661833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca20d70-e29a-42ea-b8e7-03aa0572918d None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=4d4a71b6005343cda93a92465067a7d7, role_id=68bd116b233640b7b7e07ef743a8be88)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.663291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca20d70-e29a-42ea-b8e7-03aa0572918d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.682524 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 478/951] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:10:54 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/4d4a71b6005343cda93a92465067a7d7/roles/68bd116b233640b7b7e07ef743a8be88 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:10:54.685310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca20d70-e29a-42ea-b8e7-03aa0572918d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:54.764199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17379db-f290-4e92-b167-a11f4964ede0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.807869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea40f-6bde-4bfe-b148-5dd07dc6bc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.808797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea40f-6bde-4bfe-b148-5dd07dc6bc92 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.809222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea40f-6bde-4bfe-b148-5dd07dc6bc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.809702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea40f-6bde-4bfe-b148-5dd07dc6bc92 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.810546 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea40f-6bde-4bfe-b148-5dd07dc6bc92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:54.811559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea40f-6bde-4bfe-b148-5dd07dc6bc92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.813221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea40f-6bde-4bfe-b148-5dd07dc6bc92 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.844556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 474/952] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:54.858137 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:54.863546 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca20d70-e29a-42ea-b8e7-03aa0572918d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:54.912746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74115d1b-49d4-458e-8ba1-74e772372071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:54.913370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74115d1b-49d4-458e-8ba1-74e772372071 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:54.913651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74115d1b-49d4-458e-8ba1-74e772372071 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:54.913925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74115d1b-49d4-458e-8ba1-74e772372071 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:54.914647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74115d1b-49d4-458e-8ba1-74e772372071 None admin] RBAC: Adding query filter params (interface=public, service_id=110c3f7ee62a4bf4890a30a1b7550ffd, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:54.914959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74115d1b-49d4-458e-8ba1-74e772372071 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:54.917328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74115d1b-49d4-458e-8ba1-74e772372071 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:54.923695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 479/953] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Aug 19 15:10:54 2022] GET /identity/v3/endpoints?service_id=110c3f7ee62a4bf4890a30a1b7550ffd&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:54.936928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea40f-6bde-4bfe-b148-5dd07dc6bc92 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:55.007944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:55.116801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4669d-b6bc-4b87-a1d6-dbff8f150e88 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:55.116990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4669d-b6bc-4b87-a1d6-dbff8f150e88 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:55.117263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4669d-b6bc-4b87-a1d6-dbff8f150e88 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:55.118749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 480/954] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:10:55.124620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eda6e-ce68-4cf4-8d10-07ad6dceb1e1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:55.124985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eda6e-ce68-4cf4-8d10-07ad6dceb1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:55.125259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eda6e-ce68-4cf4-8d10-07ad6dceb1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:55.140744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea40f-6bde-4bfe-b148-5dd07dc6bc92 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:55.150041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-548eda6e-ce68-4cf4-8d10-07ad6dceb1e1 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:55.152119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-548eda6e-ce68-4cf4-8d10-07ad6dceb1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:55.196827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd589d9-8994-4739-9f66-c570bbaf3927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:55.197859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd589d9-8994-4739-9f66-c570bbaf3927 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:55.198147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd589d9-8994-4739-9f66-c570bbaf3927 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:55.198419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd589d9-8994-4739-9f66-c570bbaf3927 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:55.199033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd589d9-8994-4739-9f66-c570bbaf3927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:55.200797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd589d9-8994-4739-9f66-c570bbaf3927 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:55.211992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 475/955] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Aug 19 15:10:54 2022] GET /identity/v3/services => generated 1497 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:10:55.270906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 481/956] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:55.546734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd589d9-8994-4739-9f66-c570bbaf3927 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:55.647420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:55.652287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd589d9-8994-4739-9f66-c570bbaf3927 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:55.697097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ca801-bd5c-4d34-b5ad-7ddb840283c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:55.697712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ca801-bd5c-4d34-b5ad-7ddb840283c6 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:55.697992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ca801-bd5c-4d34-b5ad-7ddb840283c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:55.698266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ca801-bd5c-4d34-b5ad-7ddb840283c6 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:55.698929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ca801-bd5c-4d34-b5ad-7ddb840283c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:55.700313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ca801-bd5c-4d34-b5ad-7ddb840283c6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:55.717479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 476/957] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:10:55 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:56.736623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc57850-d19a-4ee5-a56f-e29f3ea3a50f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:56.737198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc57850-d19a-4ee5-a56f-e29f3ea3a50f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:56.737198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc57850-d19a-4ee5-a56f-e29f3ea3a50f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:56.738770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 482/958] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:56.745076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d60bc1-f9aa-47da-ae6d-1b0ca7cc681f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:56.745592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d60bc1-f9aa-47da-ae6d-1b0ca7cc681f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:56.745679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d60bc1-f9aa-47da-ae6d-1b0ca7cc681f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:56.764824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c7d60bc1-f9aa-47da-ae6d-1b0ca7cc681f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:56.766365 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c7d60bc1-f9aa-47da-ae6d-1b0ca7cc681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:56.846497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 477/959] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:57.122299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d36d0-d710-4eab-96e1-f4c345e9f059 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.122705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d36d0-d710-4eab-96e1-f4c345e9f059 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.122900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d36d0-d710-4eab-96e1-f4c345e9f059 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.124278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 483/960] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:57.125359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d60bc1-f9aa-47da-ae6d-1b0ca7cc681f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:57.132800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9194089-87d9-4823-9902-48dc992eb8de None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.133091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9194089-87d9-4823-9902-48dc992eb8de None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.133335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9194089-87d9-4823-9902-48dc992eb8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.157393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f9194089-87d9-4823-9902-48dc992eb8de None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:57.158951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9194089-87d9-4823-9902-48dc992eb8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:57.218688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:57.239589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d60bc1-f9aa-47da-ae6d-1b0ca7cc681f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:57.251757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 484/961] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:10:57.251757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da3d5-829c-44a3-9482-21f8f988e3a9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.251757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da3d5-829c-44a3-9482-21f8f988e3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.251757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da3d5-829c-44a3-9482-21f8f988e3a9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.251757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 485/962] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:57.253222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4483d-fa16-4501-9290-2e0fa81c1489 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.253498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4483d-fa16-4501-9290-2e0fa81c1489 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.253764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4483d-fa16-4501-9290-2e0fa81c1489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.277119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5cc4483d-fa16-4501-9290-2e0fa81c1489 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:57.278328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5cc4483d-fa16-4501-9290-2e0fa81c1489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:57.288448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b970eca-9600-43bf-b353-4c72ca30d566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:57.289086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b970eca-9600-43bf-b353-4c72ca30d566 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.289363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b970eca-9600-43bf-b353-4c72ca30d566 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.289948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b970eca-9600-43bf-b353-4c72ca30d566 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.296187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:57.305220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b970eca-9600-43bf-b353-4c72ca30d566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9b970eca-9600-43bf-b353-4c72ca30d566 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:10:57.312690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:57.313800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:57.314734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 478/963] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Aug 19 15:10:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:57.321257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b970eca-9600-43bf-b353-4c72ca30d566 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:57.359016 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 486/964] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:57.432918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b970eca-9600-43bf-b353-4c72ca30d566 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:57.469019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eb60e-b5b1-4e04-b080-3cb78f91aecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:57.469945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eb60e-b5b1-4e04-b080-3cb78f91aecc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.470447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eb60e-b5b1-4e04-b080-3cb78f91aecc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.470933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eb60e-b5b1-4e04-b080-3cb78f91aecc None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.471824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eb60e-b5b1-4e04-b080-3cb78f91aecc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:57.472370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eb60e-b5b1-4e04-b080-3cb78f91aecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:57.473981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eb60e-b5b1-4e04-b080-3cb78f91aecc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:57.483569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 479/965] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Aug 19 15:10:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:57.489520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4483d-fa16-4501-9290-2e0fa81c1489 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:57.566342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eb60e-b5b1-4e04-b080-3cb78f91aecc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:57.575730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:57.600615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4483d-fa16-4501-9290-2e0fa81c1489 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:57.637993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649bb4-36c3-40f6-b93a-00ad880b350a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:57.639098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649bb4-36c3-40f6-b93a-00ad880b350a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.639598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649bb4-36c3-40f6-b93a-00ad880b350a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.640096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649bb4-36c3-40f6-b93a-00ad880b350a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.651632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649bb4-36c3-40f6-b93a-00ad880b350a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:57.653729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649bb4-36c3-40f6-b93a-00ad880b350a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:57.673299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eb60e-b5b1-4e04-b080-3cb78f91aecc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e0649bb4-36c3-40f6-b93a-00ad880b350a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:57.675800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:57.676804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:57.677748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:57.677748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:57.677748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:57.680558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 487/966] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:57.682033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649bb4-36c3-40f6-b93a-00ad880b350a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:57.686480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:57.712081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:57.713302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.713659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.713942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.714625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:57.716047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:57.730978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.sql.core [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 19 15:10:57.730978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 19 15:10:57.730978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [parameters: {'user_id': '2a6ac40aa1b64152a2b607dd57f70767', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 19 15:10:57.730978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81504) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Aug 19 15:10:57.738125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 15:10:57.739330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application yield resource Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.rollback() Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.force_reraise() Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise self.value Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application session.commit() Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 19 15:10:57.740595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._prepare_impl() Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.flush() Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._flush(objects) Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application compat.raise_( Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application flush_context.execute() Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rec.execute(self) Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 19 15:10:57.741667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 15:10:57.742647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 15:10:57.746264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [parameters: {'user_id': '2a6ac40aa1b64152a2b607dd57f70767', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:57.747306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 19 15:10:57.748325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 19 15:10:57.749968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 19 15:10:57.749968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 19 15:10:57.749968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 19 15:10:57.749968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 19 15:10:57.749968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 19 15:10:57.749968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:57.749968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 480/967] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Aug 19 15:10:57 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:57.749968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:57.797744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649bb4-36c3-40f6-b93a-00ad880b350a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:57.813235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:57.833848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a278158-dd90-40e0-b6d2-d0b46715fdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:57.834476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a278158-dd90-40e0-b6d2-d0b46715fdac None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.834748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a278158-dd90-40e0-b6d2-d0b46715fdac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.835019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a278158-dd90-40e0-b6d2-d0b46715fdac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.835596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a278158-dd90-40e0-b6d2-d0b46715fdac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:57.837027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a278158-dd90-40e0-b6d2-d0b46715fdac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0a278158-dd90-40e0-b6d2-d0b46715fdac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:57.846018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:57.847308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:57.848207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 488/968] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:10:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:57.849064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23004f26-82b8-4bc1-9d9f-122b7b4706b8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.849064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23004f26-82b8-4bc1-9d9f-122b7b4706b8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.849064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23004f26-82b8-4bc1-9d9f-122b7b4706b8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.849263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8237b8-e981-4e72-8bae-f0f8ad75fa84 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:57.850205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 489/969] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:10:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:10:57.850902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23004f26-82b8-4bc1-9d9f-122b7b4706b8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:57.886077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dba526-d381-49da-857a-c093214aefad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:57.886684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dba526-d381-49da-857a-c093214aefad None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:57.886958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dba526-d381-49da-857a-c093214aefad None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:57.887229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dba526-d381-49da-857a-c093214aefad None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:57.887864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dba526-d381-49da-857a-c093214aefad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:57.888161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dba526-d381-49da-857a-c093214aefad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:57.889775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dba526-d381-49da-857a-c093214aefad None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:57.904802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 481/970] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:57.905586 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dba526-d381-49da-857a-c093214aefad None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:57.926929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:57.965729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23004f26-82b8-4bc1-9d9f-122b7b4706b8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.000998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f7db60-ec31-4ee7-8895-51d2f5c56ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.001625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7db60-ec31-4ee7-8895-51d2f5c56ebb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.001899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7db60-ec31-4ee7-8895-51d2f5c56ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.002171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7db60-ec31-4ee7-8895-51d2f5c56ebb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.005567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dba526-d381-49da-857a-c093214aefad None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.009618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7db60-ec31-4ee7-8895-51d2f5c56ebb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.011607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f7db60-ec31-4ee7-8895-51d2f5c56ebb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-47f7db60-ec31-4ee7-8895-51d2f5c56ebb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:58.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:58.024598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:10:58.025558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 490/971] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:10:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:58.026604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aab08-0b86-488c-859a-f915898a793f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.026604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aab08-0b86-488c-859a-f915898a793f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.026801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aab08-0b86-488c-859a-f915898a793f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.035803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:58.044851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe5a14-9a99-43f9-8331-60c4cd5f2556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.045453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe5a14-9a99-43f9-8331-60c4cd5f2556 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.045575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4c5aab08-0b86-488c-859a-f915898a793f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:10:58.045730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe5a14-9a99-43f9-8331-60c4cd5f2556 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.046000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe5a14-9a99-43f9-8331-60c4cd5f2556 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.046632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe5a14-9a99-43f9-8331-60c4cd5f2556 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:58.046743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4c5aab08-0b86-488c-859a-f915898a793f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:10:58.046931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe5a14-9a99-43f9-8331-60c4cd5f2556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.048287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe5a14-9a99-43f9-8331-60c4cd5f2556 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.053197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 482/972] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:10:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:58.054156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe5a14-9a99-43f9-8331-60c4cd5f2556 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:58.114488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 491/973] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:10:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:10:58.115330 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5aab08-0b86-488c-859a-f915898a793f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:58.174016 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:58.178300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe5a14-9a99-43f9-8331-60c4cd5f2556 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.216195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be44a8-966b-43d0-8e33-f1f77a33baf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.217024 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be44a8-966b-43d0-8e33-f1f77a33baf4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.217305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be44a8-966b-43d0-8e33-f1f77a33baf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.217578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be44a8-966b-43d0-8e33-f1f77a33baf4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.218467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5aab08-0b86-488c-859a-f915898a793f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.226321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be44a8-966b-43d0-8e33-f1f77a33baf4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.227777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be44a8-966b-43d0-8e33-f1f77a33baf4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-09be44a8-966b-43d0-8e33-f1f77a33baf4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:10:58.244696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:58.245657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 483/974] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:10:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:58.251706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be44a8-966b-43d0-8e33-f1f77a33baf4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:58.254596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647f9fc-d3c4-42ed-b644-45d31f97764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.255213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647f9fc-d3c4-42ed-b644-45d31f97764b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.255496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647f9fc-d3c4-42ed-b644-45d31f97764b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.255767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647f9fc-d3c4-42ed-b644-45d31f97764b None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.256491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647f9fc-d3c4-42ed-b644-45d31f97764b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:58.256795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647f9fc-d3c4-42ed-b644-45d31f97764b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.258227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647f9fc-d3c4-42ed-b644-45d31f97764b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.294513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 492/975] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Aug 19 15:10:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:58.301772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647f9fc-d3c4-42ed-b644-45d31f97764b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:58.307049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:58.366807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be44a8-966b-43d0-8e33-f1f77a33baf4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.407789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e392d4-e414-401c-8bd9-d79cfb0d5888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.408450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e392d4-e414-401c-8bd9-d79cfb0d5888 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.408736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e392d4-e414-401c-8bd9-d79cfb0d5888 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.409008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e392d4-e414-401c-8bd9-d79cfb0d5888 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.409665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e392d4-e414-401c-8bd9-d79cfb0d5888 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:58.409970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e392d4-e414-401c-8bd9-d79cfb0d5888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.411405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e392d4-e414-401c-8bd9-d79cfb0d5888 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.423801 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 484/976] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:10:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:58.424888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e392d4-e414-401c-8bd9-d79cfb0d5888 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:58.438513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647f9fc-d3c4-42ed-b644-45d31f97764b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.458056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:58.487906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be366db3-d023-4614-bfbf-7cf7762b1fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.488492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be366db3-d023-4614-bfbf-7cf7762b1fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.488764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be366db3-d023-4614-bfbf-7cf7762b1fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.489042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be366db3-d023-4614-bfbf-7cf7762b1fa7 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.489921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be366db3-d023-4614-bfbf-7cf7762b1fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:58.490239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be366db3-d023-4614-bfbf-7cf7762b1fa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.491848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be366db3-d023-4614-bfbf-7cf7762b1fa7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.515386 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 493/977] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Aug 19 15:10:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:58.516287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be366db3-d023-4614-bfbf-7cf7762b1fa7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:58.546841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e392d4-e414-401c-8bd9-d79cfb0d5888 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.563327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:58.582045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da50126-c7b4-43ae-9653-a82a6aa1c561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.582760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da50126-c7b4-43ae-9653-a82a6aa1c561 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.582943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da50126-c7b4-43ae-9653-a82a6aa1c561 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.583213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da50126-c7b4-43ae-9653-a82a6aa1c561 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.583795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da50126-c7b4-43ae-9653-a82a6aa1c561 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.585244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da50126-c7b4-43ae-9653-a82a6aa1c561 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4da50126-c7b4-43ae-9653-a82a6aa1c561 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:58.594371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:58.595478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:58.596322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 485/978] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:10:58.597157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da50126-c7b4-43ae-9653-a82a6aa1c561 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:58.629067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be366db3-d023-4614-bfbf-7cf7762b1fa7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.663911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289ce09-2a6d-416e-ac52-8a954275ff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.664547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289ce09-2a6d-416e-ac52-8a954275ff6b None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.664825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289ce09-2a6d-416e-ac52-8a954275ff6b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.665094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289ce09-2a6d-416e-ac52-8a954275ff6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.665797 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289ce09-2a6d-416e-ac52-8a954275ff6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.667907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289ce09-2a6d-416e-ac52-8a954275ff6b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.688825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 494/979] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:10:58 2022] POST /identity/v3/endpoints => generated 349 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:10:58.689737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289ce09-2a6d-416e-ac52-8a954275ff6b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:58.706685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da50126-c7b4-43ae-9653-a82a6aa1c561 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.738045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:58.756888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcecc46-500c-4f05-acd8-44af9a6876c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.757934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcecc46-500c-4f05-acd8-44af9a6876c0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.758390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcecc46-500c-4f05-acd8-44af9a6876c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.758668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcecc46-500c-4f05-acd8-44af9a6876c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.812134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:58.836316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289ce09-2a6d-416e-ac52-8a954275ff6b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:58.840440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcecc46-500c-4f05-acd8-44af9a6876c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.841222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcecc46-500c-4f05-acd8-44af9a6876c0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.874315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a955c-d0d7-42ea-94a1-3cbd5ce4ba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:58.874929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a955c-d0d7-42ea-94a1-3cbd5ce4ba7d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:58.875229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a955c-d0d7-42ea-94a1-3cbd5ce4ba7d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:58.875670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a955c-d0d7-42ea-94a1-3cbd5ce4ba7d None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:58.876153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a955c-d0d7-42ea-94a1-3cbd5ce4ba7d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:58.876492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a955c-d0d7-42ea-94a1-3cbd5ce4ba7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:58.877890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a955c-d0d7-42ea-94a1-3cbd5ce4ba7d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:58.883163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 495/980] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:58.890223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a955c-d0d7-42ea-94a1-3cbd5ce4ba7d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:58.953493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 486/981] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:10:58 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:10:58.959065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcecc46-500c-4f05-acd8-44af9a6876c0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:59.001097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:59.008198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a955c-d0d7-42ea-94a1-3cbd5ce4ba7d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:59.046302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf1c4d-b14a-4b3a-8c7c-03609029b64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:59.046927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf1c4d-b14a-4b3a-8c7c-03609029b64f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:59.047204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf1c4d-b14a-4b3a-8c7c-03609029b64f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:59.047474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf1c4d-b14a-4b3a-8c7c-03609029b64f None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:59.048571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf1c4d-b14a-4b3a-8c7c-03609029b64f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:59.048879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf1c4d-b14a-4b3a-8c7c-03609029b64f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:59.050053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:59.050632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf1c4d-b14a-4b3a-8c7c-03609029b64f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:59.061916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 496/982] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:10:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:59.062802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf1c4d-b14a-4b3a-8c7c-03609029b64f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:59.074981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcecc46-500c-4f05-acd8-44af9a6876c0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:59.113807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778d36c-2bf5-4435-b34d-5f2d933f2495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:59.114447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778d36c-2bf5-4435-b34d-5f2d933f2495 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:59.115003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778d36c-2bf5-4435-b34d-5f2d933f2495 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:59.115277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778d36c-2bf5-4435-b34d-5f2d933f2495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:59.129688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778d36c-2bf5-4435-b34d-5f2d933f2495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:59.131796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778d36c-2bf5-4435-b34d-5f2d933f2495 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0778d36c-2bf5-4435-b34d-5f2d933f2495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:10:59.146941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:59.147962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:10:59.154549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:10:59.154549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:10:59.154549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:59.154549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 487/983] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:59.154549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778d36c-2bf5-4435-b34d-5f2d933f2495 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:59.180144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf1c4d-b14a-4b3a-8c7c-03609029b64f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:59.216034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec66b6b-7588-48a4-8917-88b39523e1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:59.216694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec66b6b-7588-48a4-8917-88b39523e1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:59.216968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec66b6b-7588-48a4-8917-88b39523e1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:59.217239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec66b6b-7588-48a4-8917-88b39523e1a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:59.217942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec66b6b-7588-48a4-8917-88b39523e1a4 None admin] RBAC: Adding query filter params (interface=public, service_id=a662c8944dc3496db0e0604efea52454, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:59.218248 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec66b6b-7588-48a4-8917-88b39523e1a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:59.219621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec66b6b-7588-48a4-8917-88b39523e1a4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:59.224913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 497/984] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Aug 19 15:10:59 2022] GET /identity/v3/endpoints?service_id=a662c8944dc3496db0e0604efea52454&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:10:59.230853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec66b6b-7588-48a4-8917-88b39523e1a4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:59.261879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:59.271104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778d36c-2bf5-4435-b34d-5f2d933f2495 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:59.300450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:59.309301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941c85f-5138-4694-b10c-3905fe79e21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:59.309907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941c85f-5138-4694-b10c-3905fe79e21e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:59.310190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941c85f-5138-4694-b10c-3905fe79e21e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:59.310565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941c85f-5138-4694-b10c-3905fe79e21e None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:59.311336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941c85f-5138-4694-b10c-3905fe79e21e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:59.311640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941c85f-5138-4694-b10c-3905fe79e21e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:59.313057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941c85f-5138-4694-b10c-3905fe79e21e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:59.330908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 488/985] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:59.332135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941c85f-5138-4694-b10c-3905fe79e21e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:59.349769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec66b6b-7588-48a4-8917-88b39523e1a4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:59.386877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b84dc-3511-47e0-a101-a621b3d7a578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:59.387644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b84dc-3511-47e0-a101-a621b3d7a578 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:59.387965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b84dc-3511-47e0-a101-a621b3d7a578 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:59.388322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b84dc-3511-47e0-a101-a621b3d7a578 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:59.388957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b84dc-3511-47e0-a101-a621b3d7a578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:59.390398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b84dc-3511-47e0-a101-a621b3d7a578 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:59.397164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 498/986] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Aug 19 15:10:59 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:10:59.448363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941c85f-5138-4694-b10c-3905fe79e21e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:59.489158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76da97a-1968-45e8-9625-7899dddfa72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:59.489799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76da97a-1968-45e8-9625-7899dddfa72b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:59.490075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76da97a-1968-45e8-9625-7899dddfa72b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:59.490346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76da97a-1968-45e8-9625-7899dddfa72b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:59.498802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76da97a-1968-45e8-9625-7899dddfa72b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:59.500293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76da97a-1968-45e8-9625-7899dddfa72b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a76da97a-1968-45e8-9625-7899dddfa72b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:10:59.513885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:10:59.514853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:10:59.521004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:10:59.521004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:10:59.521004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:10:59.521004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:10:59.521004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:10:59.521004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:10:59.521004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:10:59.521004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:10:59.521004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 489/987] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:10:59.521591 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b84dc-3511-47e0-a101-a621b3d7a578 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:59.616535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:59.632336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b84dc-3511-47e0-a101-a621b3d7a578 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:59.668837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba98843-b906-4746-9d5f-642f2139c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:59.668837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba98843-b906-4746-9d5f-642f2139c374 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:59.669134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba98843-b906-4746-9d5f-642f2139c374 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:59.669347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba98843-b906-4746-9d5f-642f2139c374 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:59.672438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba98843-b906-4746-9d5f-642f2139c374 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:59.672438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba98843-b906-4746-9d5f-642f2139c374 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:59.672438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba98843-b906-4746-9d5f-642f2139c374 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:59.687873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 499/988] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:10:59.694069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76da97a-1968-45e8-9625-7899dddfa72b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:10:59.746026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:10:59.803933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76da97a-1968-45e8-9625-7899dddfa72b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:10:59.839560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa5569-7228-450f-b53e-a37c231f41e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:10:59.840344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa5569-7228-450f-b53e-a37c231f41e4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:10:59.840754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa5569-7228-450f-b53e-a37c231f41e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:10:59.841131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa5569-7228-450f-b53e-a37c231f41e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:10:59.841966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa5569-7228-450f-b53e-a37c231f41e4 None admin] RBAC: Adding query filter params (role.id=68bd116b233640b7b7e07ef743a8be88, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=4d4a71b6005343cda93a92465067a7d7) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:10:59.842378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa5569-7228-450f-b53e-a37c231f41e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:10:59.843919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa5569-7228-450f-b53e-a37c231f41e4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:10:59.850015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 490/989] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:10:59 2022] GET /identity/v3/role_assignments?user.id=4d4a71b6005343cda93a92465067a7d7&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=68bd116b233640b7b7e07ef743a8be88 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:00.630995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a02881-66a9-44d0-b0cf-622def10b644 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:00.631553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a02881-66a9-44d0-b0cf-622def10b644 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:00.631553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a02881-66a9-44d0-b0cf-622def10b644 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:00.632935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 500/990] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:00.639192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ffb66-9b6d-4bcf-9303-659769a2519d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:00.639477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ffb66-9b6d-4bcf-9303-659769a2519d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:00.639747 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ffb66-9b6d-4bcf-9303-659769a2519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:00.658203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d31ffb66-9b6d-4bcf-9303-659769a2519d None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:00.659701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d31ffb66-9b6d-4bcf-9303-659769a2519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:00.736434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 491/991] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:01.002542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a02881-66a9-44d0-b0cf-622def10b644 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:01.109239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a02881-66a9-44d0-b0cf-622def10b644 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:01.143638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845ff3e-ae28-4f23-8906-86cf8ad7dc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:01.144277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845ff3e-ae28-4f23-8906-86cf8ad7dc83 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:01.144621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845ff3e-ae28-4f23-8906-86cf8ad7dc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:01.144891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845ff3e-ae28-4f23-8906-86cf8ad7dc83 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:01.145428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845ff3e-ae28-4f23-8906-86cf8ad7dc83 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:01.146835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5845ff3e-ae28-4f23-8906-86cf8ad7dc83 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5845ff3e-ae28-4f23-8906-86cf8ad7dc83 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:01.155873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:01.157118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:11:01.158045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:01.160729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 501/992] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:11:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:01.163002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ffb66-9b6d-4bcf-9303-659769a2519d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:01.211486 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:01.271337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ffb66-9b6d-4bcf-9303-659769a2519d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:01.306135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da111e6-8d2f-411f-b2ca-82690d85baef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:01.307507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da111e6-8d2f-411f-b2ca-82690d85baef None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:01.307849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da111e6-8d2f-411f-b2ca-82690d85baef None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:01.308215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da111e6-8d2f-411f-b2ca-82690d85baef None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:01.308960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da111e6-8d2f-411f-b2ca-82690d85baef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:01.309354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da111e6-8d2f-411f-b2ca-82690d85baef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:01.310817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da111e6-8d2f-411f-b2ca-82690d85baef None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:01.320477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 492/993] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:11:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:01.322081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845ff3e-ae28-4f23-8906-86cf8ad7dc83 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:01.335628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:01.434198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5845ff3e-ae28-4f23-8906-86cf8ad7dc83 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:01.470614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706dc9c-1b3c-47bb-b442-71b08ed777a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:01.471380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706dc9c-1b3c-47bb-b442-71b08ed777a1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:01.471755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706dc9c-1b3c-47bb-b442-71b08ed777a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:01.472124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706dc9c-1b3c-47bb-b442-71b08ed777a1 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:01.472886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706dc9c-1b3c-47bb-b442-71b08ed777a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:01.473307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706dc9c-1b3c-47bb-b442-71b08ed777a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:01.474765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706dc9c-1b3c-47bb-b442-71b08ed777a1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:01.480684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 502/994] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:11:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:01.486419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da111e6-8d2f-411f-b2ca-82690d85baef None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:01.487029 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f40a2-cc46-4e56-8762-67b6c6a80fc9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:01.487419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f40a2-cc46-4e56-8762-67b6c6a80fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:01.487792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f40a2-cc46-4e56-8762-67b6c6a80fc9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:01.489198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 503/995] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:01.494802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c8785-3b29-4fbb-a0c3-5071192f30f0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:01.495439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c8785-3b29-4fbb-a0c3-5071192f30f0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:01.495732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c8785-3b29-4fbb-a0c3-5071192f30f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:01.514925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b37c8785-3b29-4fbb-a0c3-5071192f30f0 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:01.516682 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b37c8785-3b29-4fbb-a0c3-5071192f30f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:01.594200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 504/996] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:01.608719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da111e6-8d2f-411f-b2ca-82690d85baef None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:01.615987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:01.651516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49c36a-361a-4460-b02f-8650d4ed7fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:01.652681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c36a-361a-4460-b02f-8650d4ed7fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:01.652967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c36a-361a-4460-b02f-8650d4ed7fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:01.653231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c36a-361a-4460-b02f-8650d4ed7fc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:01.654347 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49c36a-361a-4460-b02f-8650d4ed7fc7 None admin] RBAC: Adding query filter params (interface=admin, service_id=110c3f7ee62a4bf4890a30a1b7550ffd, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:01.654715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49c36a-361a-4460-b02f-8650d4ed7fc7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:01.656201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49c36a-361a-4460-b02f-8650d4ed7fc7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:01.668452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 493/997] 10.222.0.75 () {56 vars in 1249 bytes} [Fri Aug 19 15:11:01 2022] GET /identity/v3/endpoints?service_id=110c3f7ee62a4bf4890a30a1b7550ffd&interface=admin®ion_id=RegionOne => generated 193 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:01.671350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c8785-3b29-4fbb-a0c3-5071192f30f0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:01.678523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:01.781410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c8785-3b29-4fbb-a0c3-5071192f30f0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:01.793305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7757-5919-42f6-a28e-ba441cb69854 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:01.793616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7757-5919-42f6-a28e-ba441cb69854 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:01.793859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7757-5919-42f6-a28e-ba441cb69854 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:01.795448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 494/998] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:11:01.800329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccd21-4efb-49ca-8365-3963ac31c064 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:01.800634 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccd21-4efb-49ca-8365-3963ac31c064 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:01.800917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccd21-4efb-49ca-8365-3963ac31c064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:01.818263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fd158-b67a-41b0-80c9-5fb25eda1707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:01.818867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fd158-b67a-41b0-80c9-5fb25eda1707 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:01.819148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fd158-b67a-41b0-80c9-5fb25eda1707 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:01.819421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fd158-b67a-41b0-80c9-5fb25eda1707 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:01.819528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-59cccd21-4efb-49ca-8365-3963ac31c064 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:01.820028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fd158-b67a-41b0-80c9-5fb25eda1707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:01.820673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-59cccd21-4efb-49ca-8365-3963ac31c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:01.821486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fd158-b67a-41b0-80c9-5fb25eda1707 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:01.836492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 505/999] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Aug 19 15:11:01 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:11:01.871358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fd158-b67a-41b0-80c9-5fb25eda1707 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:01.906293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 495/1000] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:01.946415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:01.983703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fd158-b67a-41b0-80c9-5fb25eda1707 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:02.019949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90330889-8de9-49f5-a56b-4b7396f357bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:02.020889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90330889-8de9-49f5-a56b-4b7396f357bd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:02.021167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90330889-8de9-49f5-a56b-4b7396f357bd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:02.021437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90330889-8de9-49f5-a56b-4b7396f357bd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:02.022015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90330889-8de9-49f5-a56b-4b7396f357bd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:02.023391 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90330889-8de9-49f5-a56b-4b7396f357bd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-90330889-8de9-49f5-a56b-4b7396f357bd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:02.032930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:02.033950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:02.034791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 506/1001] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:11:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:02.042352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccd21-4efb-49ca-8365-3963ac31c064 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:02.148728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:02.152954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccd21-4efb-49ca-8365-3963ac31c064 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:02.177654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90330889-8de9-49f5-a56b-4b7396f357bd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:02.191103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103c0e0-586d-48dc-95a4-17dbbcf011ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:02.191767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103c0e0-586d-48dc-95a4-17dbbcf011ed None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:02.191983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103c0e0-586d-48dc-95a4-17dbbcf011ed None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:02.192252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103c0e0-586d-48dc-95a4-17dbbcf011ed None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:02.192917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103c0e0-586d-48dc-95a4-17dbbcf011ed None admin] RBAC: Adding query filter params (name=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:02.193224 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103c0e0-586d-48dc-95a4-17dbbcf011ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:02.194599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103c0e0-586d-48dc-95a4-17dbbcf011ed None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:02.199960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 496/1002] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:11:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:02.205397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103c0e0-586d-48dc-95a4-17dbbcf011ed None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:02.285502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90330889-8de9-49f5-a56b-4b7396f357bd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:02.318252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:02.323960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103c0e0-586d-48dc-95a4-17dbbcf011ed None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:02.324514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eae3f5-4bb4-42ac-8913-f26421169391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:02.325134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eae3f5-4bb4-42ac-8913-f26421169391 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:02.325412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eae3f5-4bb4-42ac-8913-f26421169391 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:02.325687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eae3f5-4bb4-42ac-8913-f26421169391 None admin] PATH_INFO: `/v3/services/image` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:02.326267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eae3f5-4bb4-42ac-8913-f26421169391 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:02.327654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eae3f5-4bb4-42ac-8913-f26421169391 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-57eae3f5-4bb4-42ac-8913-f26421169391 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:02.336572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:02.337663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:02.339661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 507/1003] 10.222.0.75 () {56 vars in 1105 bytes} [Fri Aug 19 15:11:02 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:02.344016 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eae3f5-4bb4-42ac-8913-f26421169391 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:02.373848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c10c3-27ee-4293-8202-af62d61be781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:02.374457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c10c3-27ee-4293-8202-af62d61be781 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:02.374725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c10c3-27ee-4293-8202-af62d61be781 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:02.375002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c10c3-27ee-4293-8202-af62d61be781 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:02.375661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c10c3-27ee-4293-8202-af62d61be781 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:02.375940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c10c3-27ee-4293-8202-af62d61be781 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:02.377868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c10c3-27ee-4293-8202-af62d61be781 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:02.382783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 497/1004] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Aug 19 15:11:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:02.389191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c10c3-27ee-4293-8202-af62d61be781 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:02.419209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:02.461241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eae3f5-4bb4-42ac-8913-f26421169391 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:02.497505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b66bc-bd86-40ac-ab90-a3d77e1b33a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:02.498203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c10c3-27ee-4293-8202-af62d61be781 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:02.498315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b66bc-bd86-40ac-ab90-a3d77e1b33a6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:02.498576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b66bc-bd86-40ac-ab90-a3d77e1b33a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:02.498920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b66bc-bd86-40ac-ab90-a3d77e1b33a6 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:02.499626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b66bc-bd86-40ac-ab90-a3d77e1b33a6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:02.499995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b66bc-bd86-40ac-ab90-a3d77e1b33a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:02.501408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b66bc-bd86-40ac-ab90-a3d77e1b33a6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:02.506514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 508/1005] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:02.512869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b66bc-bd86-40ac-ab90-a3d77e1b33a6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:02.540746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7942c-0167-46c2-8d24-8a62251b3873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:02.541886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7942c-0167-46c2-8d24-8a62251b3873 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:02.542221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7942c-0167-46c2-8d24-8a62251b3873 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:02.542488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7942c-0167-46c2-8d24-8a62251b3873 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:02.543258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7942c-0167-46c2-8d24-8a62251b3873 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:02.544668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7942c-0167-46c2-8d24-8a62251b3873 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:02.565672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 498/1006] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:11:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:11:02.589636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:02.638388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b66bc-bd86-40ac-ab90-a3d77e1b33a6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:02.692941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae84cc-5989-4a95-8e5a-95a721d6b815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:02.694351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae84cc-5989-4a95-8e5a-95a721d6b815 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:02.694825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae84cc-5989-4a95-8e5a-95a721d6b815 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:02.695341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae84cc-5989-4a95-8e5a-95a721d6b815 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:02.696187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae84cc-5989-4a95-8e5a-95a721d6b815 None admin] RBAC: Adding query filter params (type=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:02.696725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae84cc-5989-4a95-8e5a-95a721d6b815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:02.698992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae84cc-5989-4a95-8e5a-95a721d6b815 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:02.705043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 509/1007] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:03.769492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77b3ab-e821-4296-9590-050ad3e6ec98 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:03.770061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77b3ab-e821-4296-9590-050ad3e6ec98 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:03.770061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77b3ab-e821-4296-9590-050ad3e6ec98 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:03.771391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 499/1008] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:03.778548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fcf8e-b41e-4bc4-907f-b93f3dfa20a2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:03.779019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fcf8e-b41e-4bc4-907f-b93f3dfa20a2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:03.779204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fcf8e-b41e-4bc4-907f-b93f3dfa20a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:03.800056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6e0fcf8e-b41e-4bc4-907f-b93f3dfa20a2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:03.801347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6e0fcf8e-b41e-4bc4-907f-b93f3dfa20a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:03.876044 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 510/1009] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:03 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:04.146268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77b3ab-e821-4296-9590-050ad3e6ec98 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:04.239127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:04.258370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77b3ab-e821-4296-9590-050ad3e6ec98 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:04.293076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414217b6-3a19-42ce-b556-a0c81744a87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:04.293707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414217b6-3a19-42ce-b556-a0c81744a87b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:04.293987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414217b6-3a19-42ce-b556-a0c81744a87b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:04.294259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414217b6-3a19-42ce-b556-a0c81744a87b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:04.294846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414217b6-3a19-42ce-b556-a0c81744a87b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:04.296225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414217b6-3a19-42ce-b556-a0c81744a87b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-414217b6-3a19-42ce-b556-a0c81744a87b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:04.307719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:04.309104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 15:11:04.310001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:04.312543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 500/1010] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Aug 19 15:11:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:04.315106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fcf8e-b41e-4bc4-907f-b93f3dfa20a2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:04.368191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:04.419662 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fcf8e-b41e-4bc4-907f-b93f3dfa20a2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:04.454667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d10984-38e8-4e9d-a72f-7cc9798ebf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:04.455276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d10984-38e8-4e9d-a72f-7cc9798ebf95 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:04.455555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d10984-38e8-4e9d-a72f-7cc9798ebf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:04.455828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d10984-38e8-4e9d-a72f-7cc9798ebf95 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:04.456482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d10984-38e8-4e9d-a72f-7cc9798ebf95 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:04.456786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d10984-38e8-4e9d-a72f-7cc9798ebf95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:04.458189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d10984-38e8-4e9d-a72f-7cc9798ebf95 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:04.463249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 511/1011] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:11:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:04.468938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414217b6-3a19-42ce-b556-a0c81744a87b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:04.575845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414217b6-3a19-42ce-b556-a0c81744a87b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:04.610817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6f29d-7862-4585-a22f-772687b2e906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:04.611432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6f29d-7862-4585-a22f-772687b2e906 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:04.611711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6f29d-7862-4585-a22f-772687b2e906 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:04.611983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6f29d-7862-4585-a22f-772687b2e906 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:04.612641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6f29d-7862-4585-a22f-772687b2e906 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:04.612944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6f29d-7862-4585-a22f-772687b2e906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:04.614326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6f29d-7862-4585-a22f-772687b2e906 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:04.617045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a08263-885a-487a-9a3c-0003e7c8a7a9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:04.617329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a08263-885a-487a-9a3c-0003e7c8a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:04.617598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a08263-885a-487a-9a3c-0003e7c8a7a9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:04.619614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 512/1012] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:04.626117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 501/1013] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Aug 19 15:11:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:04.627155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87773eb-d2c3-41dc-8273-6f7bd92609a2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:04.627445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87773eb-d2c3-41dc-8273-6f7bd92609a2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:04.627724 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87773eb-d2c3-41dc-8273-6f7bd92609a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:04.628165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6f29d-7862-4585-a22f-772687b2e906 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:04.644080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:04.649560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b87773eb-d2c3-41dc-8273-6f7bd92609a2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:04.650932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b87773eb-d2c3-41dc-8273-6f7bd92609a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:04.722489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 513/1014] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:04 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:04.724148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c511091-65bd-4ab8-995e-77dc55dd8fc8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:04.724517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c511091-65bd-4ab8-995e-77dc55dd8fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:04.724858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c511091-65bd-4ab8-995e-77dc55dd8fc8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:04.726338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 514/1015] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:04.731595 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076e24-a718-43da-a97c-68436fec955c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:04.731872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076e24-a718-43da-a97c-68436fec955c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:04.732145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076e24-a718-43da-a97c-68436fec955c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:04.736212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6f29d-7862-4585-a22f-772687b2e906 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:04.750933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d3076e24-a718-43da-a97c-68436fec955c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:04.752114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d3076e24-a718-43da-a97c-68436fec955c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:04.776751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3dc4e-ee40-4eb2-b5a1-bb39039e529f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:04.777753 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3dc4e-ee40-4eb2-b5a1-bb39039e529f None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:04.777961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3dc4e-ee40-4eb2-b5a1-bb39039e529f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:04.778237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3dc4e-ee40-4eb2-b5a1-bb39039e529f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:04.778903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3dc4e-ee40-4eb2-b5a1-bb39039e529f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:04.780689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3dc4e-ee40-4eb2-b5a1-bb39039e529f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:04.808323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 502/1016] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:11:04 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:11:04.809021 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:04.846288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 515/1017] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:04 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:05.009955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3dc4e-ee40-4eb2-b5a1-bb39039e529f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:05.088765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:05.123074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3dc4e-ee40-4eb2-b5a1-bb39039e529f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:05.124098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076e24-a718-43da-a97c-68436fec955c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:05.160757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f0d76-a259-4ad8-bd82-0c760624ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:05.161390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f0d76-a259-4ad8-bd82-0c760624ebed None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:05.161673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f0d76-a259-4ad8-bd82-0c760624ebed None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:05.161942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f0d76-a259-4ad8-bd82-0c760624ebed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:05.176563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f0d76-a259-4ad8-bd82-0c760624ebed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-095f0d76-a259-4ad8-bd82-0c760624ebed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:11:05.181740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:05.182694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:05.184177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 503/1018] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Aug 19 15:11:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:05.188664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f0d76-a259-4ad8-bd82-0c760624ebed None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:05.237530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076e24-a718-43da-a97c-68436fec955c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:05.275923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3704-d9f7-4f6a-b7a0-7784ba4f5bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:05.276437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3704-d9f7-4f6a-b7a0-7784ba4f5bfc None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:05.276708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3704-d9f7-4f6a-b7a0-7784ba4f5bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:05.277024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f3704-d9f7-4f6a-b7a0-7784ba4f5bfc None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:05.277894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f3704-d9f7-4f6a-b7a0-7784ba4f5bfc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:05.279274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f3704-d9f7-4f6a-b7a0-7784ba4f5bfc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:05.300574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 516/1019] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:11:05 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:11:05.303417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f0d76-a259-4ad8-bd82-0c760624ebed None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:05.341505 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:05.356158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b13e7f-cdd6-4f00-96cf-db4665bfe7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:05.356805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b13e7f-cdd6-4f00-96cf-db4665bfe7de None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:05.357171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b13e7f-cdd6-4f00-96cf-db4665bfe7de None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:05.357445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b13e7f-cdd6-4f00-96cf-db4665bfe7de None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:05.358120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b13e7f-cdd6-4f00-96cf-db4665bfe7de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:05.358420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b13e7f-cdd6-4f00-96cf-db4665bfe7de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:05.359888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b13e7f-cdd6-4f00-96cf-db4665bfe7de None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:05.376491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 504/1020] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Aug 19 15:11:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:05.377925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3704-d9f7-4f6a-b7a0-7784ba4f5bfc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:05.413635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:05.494875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f3704-d9f7-4f6a-b7a0-7784ba4f5bfc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:05.529314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12c030-eac4-452f-ab3d-bbee3d987590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:05.529868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12c030-eac4-452f-ab3d-bbee3d987590 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:05.530145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12c030-eac4-452f-ab3d-bbee3d987590 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:05.530414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12c030-eac4-452f-ab3d-bbee3d987590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:05.541493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12c030-eac4-452f-ab3d-bbee3d987590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:05.542942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12c030-eac4-452f-ab3d-bbee3d987590 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-fc12c030-eac4-452f-ab3d-bbee3d987590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:11:05.557978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:05.558953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:11:05.559853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:11:05.559853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:05.559853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:05.559853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 517/1021] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:05.565759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b13e7f-cdd6-4f00-96cf-db4665bfe7de None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:05.682100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b13e7f-cdd6-4f00-96cf-db4665bfe7de None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:05.718873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22554de3-a81a-4ac6-8389-9366ef1368f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:05.719530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22554de3-a81a-4ac6-8389-9366ef1368f0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:05.719811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22554de3-a81a-4ac6-8389-9366ef1368f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:05.720083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22554de3-a81a-4ac6-8389-9366ef1368f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:05.720755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22554de3-a81a-4ac6-8389-9366ef1368f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:05.721055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22554de3-a81a-4ac6-8389-9366ef1368f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:05.722429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22554de3-a81a-4ac6-8389-9366ef1368f0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:05.739067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 505/1022] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:05.740349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12c030-eac4-452f-ab3d-bbee3d987590 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:05.828889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:05.849389 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12c030-eac4-452f-ab3d-bbee3d987590 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:05.885692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce70c2-174c-4474-b153-a529222a4fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:05.886324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce70c2-174c-4474-b153-a529222a4fed None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:05.886600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce70c2-174c-4474-b153-a529222a4fed None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:05.886883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce70c2-174c-4474-b153-a529222a4fed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:05.894648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce70c2-174c-4474-b153-a529222a4fed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:05.896105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce70c2-174c-4474-b153-a529222a4fed None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c1ce70c2-174c-4474-b153-a529222a4fed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:11:05.907740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:05.908757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:11:05.909713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:05.910889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 518/1023] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Aug 19 15:11:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:05.914761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22554de3-a81a-4ac6-8389-9366ef1368f0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:05.969776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:06.022427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22554de3-a81a-4ac6-8389-9366ef1368f0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:06.056527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccd662-ff3e-4ef3-9f71-33ba7a902c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:06.057130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccd662-ff3e-4ef3-9f71-33ba7a902c16 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:06.057417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccd662-ff3e-4ef3-9f71-33ba7a902c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:06.057690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccd662-ff3e-4ef3-9f71-33ba7a902c16 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:06.058412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccd662-ff3e-4ef3-9f71-33ba7a902c16 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:06.058743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccd662-ff3e-4ef3-9f71-33ba7a902c16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:06.060449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccd662-ff3e-4ef3-9f71-33ba7a902c16 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:06.082327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 506/1024] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Aug 19 15:11:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:06.083576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce70c2-174c-4474-b153-a529222a4fed None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:06.186993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce70c2-174c-4474-b153-a529222a4fed None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:06.223001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d91428-adf4-4dba-aa00-51c13442c002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:06.223612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d91428-adf4-4dba-aa00-51c13442c002 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:06.224587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d91428-adf4-4dba-aa00-51c13442c002 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:06.224587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d91428-adf4-4dba-aa00-51c13442c002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:06.237760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:06.297186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d91428-adf4-4dba-aa00-51c13442c002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:06.298655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d91428-adf4-4dba-aa00-51c13442c002 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:06.404941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 519/1025] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:06 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:11:06.406100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccd662-ff3e-4ef3-9f71-33ba7a902c16 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:06.509843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccd662-ff3e-4ef3-9f71-33ba7a902c16 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:06.519136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:06.545759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee160bc6-2ca1-4570-ad77-45abc514ea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:06.546384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee160bc6-2ca1-4570-ad77-45abc514ea7d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:06.546664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee160bc6-2ca1-4570-ad77-45abc514ea7d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:06.546939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee160bc6-2ca1-4570-ad77-45abc514ea7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:06.557526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee160bc6-2ca1-4570-ad77-45abc514ea7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:06.559542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee160bc6-2ca1-4570-ad77-45abc514ea7d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ee160bc6-2ca1-4570-ad77-45abc514ea7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:11:06.574509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:06.575497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:11:06.581045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:11:06.581045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:06.581045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:06.581045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 507/1026] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:06.581285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d91428-adf4-4dba-aa00-51c13442c002 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:06.625109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:06.690500 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d91428-adf4-4dba-aa00-51c13442c002 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:06.727096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95568cbb-f400-4f25-9054-34eb0499d34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:06.727701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568cbb-f400-4f25-9054-34eb0499d34a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:06.727976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568cbb-f400-4f25-9054-34eb0499d34a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:06.728925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95568cbb-f400-4f25-9054-34eb0499d34a None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:06.730001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95568cbb-f400-4f25-9054-34eb0499d34a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:06.730306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95568cbb-f400-4f25-9054-34eb0499d34a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:06.731681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95568cbb-f400-4f25-9054-34eb0499d34a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:06.747654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 520/1027] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:06.748901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee160bc6-2ca1-4570-ad77-45abc514ea7d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:06.777583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d9211-c4af-4c8d-a009-0f2636e56449 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:06.777908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d9211-c4af-4c8d-a009-0f2636e56449 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:06.778122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d9211-c4af-4c8d-a009-0f2636e56449 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:06.779703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 521/1028] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:06.784284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cca6cb-c936-498e-a05c-9f13c45d63bd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:06.784599 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cca6cb-c936-498e-a05c-9f13c45d63bd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:06.784869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cca6cb-c936-498e-a05c-9f13c45d63bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:06.804407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-83cca6cb-c936-498e-a05c-9f13c45d63bd None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:06.805590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-83cca6cb-c936-498e-a05c-9f13c45d63bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:06.859359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee160bc6-2ca1-4570-ad77-45abc514ea7d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:06.880348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 522/1029] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:06.894954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654fe0a-01b3-45d5-b235-4e153192e0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:06.895581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654fe0a-01b3-45d5-b235-4e153192e0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:06.895868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654fe0a-01b3-45d5-b235-4e153192e0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:06.896143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654fe0a-01b3-45d5-b235-4e153192e0e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:06.903590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654fe0a-01b3-45d5-b235-4e153192e0e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:06.905082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654fe0a-01b3-45d5-b235-4e153192e0e6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6654fe0a-01b3-45d5-b235-4e153192e0e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:11:06.917047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:06.923334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:06.924499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:11:06.924499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:11:06.924499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:11:06.924499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:11:06.924499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:11:06.924499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:11:06.924499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:11:06.924499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:06.924499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 508/1030] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:06.925108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cca6cb-c936-498e-a05c-9f13c45d63bd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:06.955892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:07.032319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cca6cb-c936-498e-a05c-9f13c45d63bd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:07.067593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59300c6-3fa7-43b9-896c-a409bc301ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:07.068198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59300c6-3fa7-43b9-896c-a409bc301ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:07.068508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59300c6-3fa7-43b9-896c-a409bc301ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:07.068784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59300c6-3fa7-43b9-896c-a409bc301ce4 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:07.069478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59300c6-3fa7-43b9-896c-a409bc301ce4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:07.069786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59300c6-3fa7-43b9-896c-a409bc301ce4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:07.071229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59300c6-3fa7-43b9-896c-a409bc301ce4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:07.089304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 523/1031] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:07.090581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654fe0a-01b3-45d5-b235-4e153192e0e6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:07.113705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:07.174436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59300c6-3fa7-43b9-896c-a409bc301ce4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:07.196761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654fe0a-01b3-45d5-b235-4e153192e0e6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:07.233524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b1b4e-32e1-4b79-bb52-cf32a11ea629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:07.234178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b1b4e-32e1-4b79-bb52-cf32a11ea629 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:07.234393 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b1b4e-32e1-4b79-bb52-cf32a11ea629 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:07.234660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b1b4e-32e1-4b79-bb52-cf32a11ea629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:07.235388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b1b4e-32e1-4b79-bb52-cf32a11ea629 None admin] RBAC: Adding query filter params (role.id=68bd116b233640b7b7e07ef743a8be88, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=98a63d5b605a4ecfad2e305c491ae755) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:07.235693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b1b4e-32e1-4b79-bb52-cf32a11ea629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:07.237148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b1b4e-32e1-4b79-bb52-cf32a11ea629 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:07.252456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 509/1032] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:11:07 2022] GET /identity/v3/role_assignments?user.id=98a63d5b605a4ecfad2e305c491ae755&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=68bd116b233640b7b7e07ef743a8be88 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:07.259989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeb729-2071-4a33-bb71-5664bb26d650 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:07.260295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeb729-2071-4a33-bb71-5664bb26d650 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:07.260585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeb729-2071-4a33-bb71-5664bb26d650 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:07.263191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 510/1033] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:11:07.268793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf357c-7f5e-46f6-a477-03adb1eeb2d8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:07.269083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf357c-7f5e-46f6-a477-03adb1eeb2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:07.269362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf357c-7f5e-46f6-a477-03adb1eeb2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:07.273427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:07.284458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59300c6-3fa7-43b9-896c-a409bc301ce4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:07.292562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e5cf357c-7f5e-46f6-a477-03adb1eeb2d8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:07.293822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e5cf357c-7f5e-46f6-a477-03adb1eeb2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:07.320894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7d7b3-631b-4e9e-b7e0-86362cac0f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:07.321548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7d7b3-631b-4e9e-b7e0-86362cac0f0e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:07.321833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7d7b3-631b-4e9e-b7e0-86362cac0f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:07.322654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7d7b3-631b-4e9e-b7e0-86362cac0f0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:07.323932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7d7b3-631b-4e9e-b7e0-86362cac0f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:07.325460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7d7b3-631b-4e9e-b7e0-86362cac0f0e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:07.327172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 524/1034] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:07.333223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7d7b3-631b-4e9e-b7e0-86362cac0f0e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:07.374381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 511/1035] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:07.455870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7d7b3-631b-4e9e-b7e0-86362cac0f0e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:07.490983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15e2cc-15cd-4143-8933-93c0f29ab08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:07.491585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15e2cc-15cd-4143-8933-93c0f29ab08e None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:07.491860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15e2cc-15cd-4143-8933-93c0f29ab08e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:07.492130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15e2cc-15cd-4143-8933-93c0f29ab08e None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:07.492790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15e2cc-15cd-4143-8933-93c0f29ab08e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:07.494891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15e2cc-15cd-4143-8933-93c0f29ab08e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:07.528427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 525/1036] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:11:07 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:11:07.643477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf357c-7f5e-46f6-a477-03adb1eeb2d8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:07.659943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:07.765471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf357c-7f5e-46f6-a477-03adb1eeb2d8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:07.800783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6f9e6-31c3-43be-96f4-02d7f0d735bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:07.801645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6f9e6-31c3-43be-96f4-02d7f0d735bb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:07.802101 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6f9e6-31c3-43be-96f4-02d7f0d735bb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:07.802518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6f9e6-31c3-43be-96f4-02d7f0d735bb None admin] PATH_INFO: `/v3/services/image` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:07.803301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6f9e6-31c3-43be-96f4-02d7f0d735bb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:07.804901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6f9e6-31c3-43be-96f4-02d7f0d735bb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-25c6f9e6-31c3-43be-96f4-02d7f0d735bb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:07.815402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:07.816638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 15:11:07.817698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:07.819760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 512/1037] 10.222.0.75 () {56 vars in 1105 bytes} [Fri Aug 19 15:11:07 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:07.826218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15e2cc-15cd-4143-8933-93c0f29ab08e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:07.843307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:07.935669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15e2cc-15cd-4143-8933-93c0f29ab08e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:07.969919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ec190-4b2a-408c-aa3a-336fc96a527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:07.970525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ec190-4b2a-408c-aa3a-336fc96a527c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:07.970803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ec190-4b2a-408c-aa3a-336fc96a527c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:07.971074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ec190-4b2a-408c-aa3a-336fc96a527c None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:07.971700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ec190-4b2a-408c-aa3a-336fc96a527c None admin] RBAC: Adding query filter params (name=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:07.971999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ec190-4b2a-408c-aa3a-336fc96a527c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:07.973401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ec190-4b2a-408c-aa3a-336fc96a527c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:07.979032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 526/1038] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:07.984852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6f9e6-31c3-43be-96f4-02d7f0d735bb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:08.071688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:08.095117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6f9e6-31c3-43be-96f4-02d7f0d735bb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:08.129720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58acfc5-2a4a-4760-ba27-dc84c50976a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:08.130320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58acfc5-2a4a-4760-ba27-dc84c50976a7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:08.130597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58acfc5-2a4a-4760-ba27-dc84c50976a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:08.130866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58acfc5-2a4a-4760-ba27-dc84c50976a7 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:08.131495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58acfc5-2a4a-4760-ba27-dc84c50976a7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:08.131792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58acfc5-2a4a-4760-ba27-dc84c50976a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:08.133195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58acfc5-2a4a-4760-ba27-dc84c50976a7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:08.142936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 513/1039] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:08.144316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ec190-4b2a-408c-aa3a-336fc96a527c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:08.227495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:08.252835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ec190-4b2a-408c-aa3a-336fc96a527c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:08.289227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908472c-fa5b-4c8d-91ee-ad67ac1db848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:08.289854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908472c-fa5b-4c8d-91ee-ad67ac1db848 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:08.290138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908472c-fa5b-4c8d-91ee-ad67ac1db848 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:08.290410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908472c-fa5b-4c8d-91ee-ad67ac1db848 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:08.291742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908472c-fa5b-4c8d-91ee-ad67ac1db848 None admin] RBAC: Adding query filter params (interface=public, service_id=f38360332d2c4260b32e4d3545f0876f, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:08.292029 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908472c-fa5b-4c8d-91ee-ad67ac1db848 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:08.293442 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908472c-fa5b-4c8d-91ee-ad67ac1db848 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:08.303163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 527/1040] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Aug 19 15:11:08 2022] GET /identity/v3/endpoints?service_id=f38360332d2c4260b32e4d3545f0876f&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:08.304762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58acfc5-2a4a-4760-ba27-dc84c50976a7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:08.409995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58acfc5-2a4a-4760-ba27-dc84c50976a7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:08.443485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c80c2-a6e1-4ff5-af61-b3dd36a60054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:08.444081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c80c2-a6e1-4ff5-af61-b3dd36a60054 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:08.444355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c80c2-a6e1-4ff5-af61-b3dd36a60054 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:08.444652 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c80c2-a6e1-4ff5-af61-b3dd36a60054 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:08.445245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c80c2-a6e1-4ff5-af61-b3dd36a60054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:08.446602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c80c2-a6e1-4ff5-af61-b3dd36a60054 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:08.460484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 514/1041] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Aug 19 15:11:08 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:11:09.185114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12e9e-e42f-47e8-bf8d-92630cf9b5e4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:09.185647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12e9e-e42f-47e8-bf8d-92630cf9b5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:09.185647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a12e9e-e42f-47e8-bf8d-92630cf9b5e4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:09.187171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 528/1042] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:09.192319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bd5d6-cc43-4f2b-9730-65ec5323a69c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:09.192775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bd5d6-cc43-4f2b-9730-65ec5323a69c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:09.192946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bd5d6-cc43-4f2b-9730-65ec5323a69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:09.205707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:09.212202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a67bd5d6-cc43-4f2b-9730-65ec5323a69c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:09.214618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a67bd5d6-cc43-4f2b-9730-65ec5323a69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:09.282848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 515/1043] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:09.504681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dcd7d-ebd4-4152-8f96-ca9b99f009db None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:09.505020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dcd7d-ebd4-4152-8f96-ca9b99f009db None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:09.505199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dcd7d-ebd4-4152-8f96-ca9b99f009db None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:09.506600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 529/1044] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:09.512088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc24a82-dfae-4b89-8ded-d7625ebee81f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:09.512378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc24a82-dfae-4b89-8ded-d7625ebee81f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:09.512678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc24a82-dfae-4b89-8ded-d7625ebee81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:09.532219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-acc24a82-dfae-4b89-8ded-d7625ebee81f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:09.533467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-acc24a82-dfae-4b89-8ded-d7625ebee81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:09.549548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dcd7d-ebd4-4152-8f96-ca9b99f009db None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:09.602737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 516/1045] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:09.660375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dcd7d-ebd4-4152-8f96-ca9b99f009db None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:09.699624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b30f01-cb48-42cb-8139-a63ac32a894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:09.700260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b30f01-cb48-42cb-8139-a63ac32a894d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:09.700563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b30f01-cb48-42cb-8139-a63ac32a894d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:09.700835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b30f01-cb48-42cb-8139-a63ac32a894d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:09.709154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:09.712308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b30f01-cb48-42cb-8139-a63ac32a894d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-32b30f01-cb48-42cb-8139-a63ac32a894d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:11:09.717828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:09.718988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:11:09.720066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:09.723132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 530/1046] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Aug 19 15:11:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:09.728908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc24a82-dfae-4b89-8ded-d7625ebee81f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:09.797230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:09.833252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc24a82-dfae-4b89-8ded-d7625ebee81f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:09.870567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed9020-5b91-48f5-8817-1571f92b755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:09.871175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed9020-5b91-48f5-8817-1571f92b755f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:09.871453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed9020-5b91-48f5-8817-1571f92b755f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:09.871723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed9020-5b91-48f5-8817-1571f92b755f None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:09.872400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed9020-5b91-48f5-8817-1571f92b755f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:09.872709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed9020-5b91-48f5-8817-1571f92b755f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:09.874084 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed9020-5b91-48f5-8817-1571f92b755f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:09.882610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 517/1047] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Aug 19 15:11:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:09.883699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b30f01-cb48-42cb-8139-a63ac32a894d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:09.888325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed9020-5b91-48f5-8817-1571f92b755f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:09.994954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b30f01-cb48-42cb-8139-a63ac32a894d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:09.995891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed9020-5b91-48f5-8817-1571f92b755f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:10.026452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:10.034539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4826e92-01ad-4258-8f1a-585a71655827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:10.035168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4826e92-01ad-4258-8f1a-585a71655827 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.035446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4826e92-01ad-4258-8f1a-585a71655827 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.035646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301049c1-7b2e-4e81-8567-173989a1bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:10.035740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4826e92-01ad-4258-8f1a-585a71655827 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.036211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301049c1-7b2e-4e81-8567-173989a1bd0a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.036444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301049c1-7b2e-4e81-8567-173989a1bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.036718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301049c1-7b2e-4e81-8567-173989a1bd0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.036833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4826e92-01ad-4258-8f1a-585a71655827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:10.038382 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4826e92-01ad-4258-8f1a-585a71655827 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:10.040420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 531/1048] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:10.046314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4826e92-01ad-4258-8f1a-585a71655827 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:10.049489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301049c1-7b2e-4e81-8567-173989a1bd0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:10.050886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301049c1-7b2e-4e81-8567-173989a1bd0a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-301049c1-7b2e-4e81-8567-173989a1bd0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:11:10.067208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:10.068223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:11:10.074459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:11:10.074459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:10.074459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:10.074459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 518/1049] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:10.074459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301049c1-7b2e-4e81-8567-173989a1bd0a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:10.122320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:10.160039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4826e92-01ad-4258-8f1a-585a71655827 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:10.187183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301049c1-7b2e-4e81-8567-173989a1bd0a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:10.198245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefea33-e24b-4029-865e-9c7f31b138ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:10.198947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefea33-e24b-4029-865e-9c7f31b138ac None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.199229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefea33-e24b-4029-865e-9c7f31b138ac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.199509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefea33-e24b-4029-865e-9c7f31b138ac None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.200157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefea33-e24b-4029-865e-9c7f31b138ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:10.201861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefea33-e24b-4029-865e-9c7f31b138ac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:10.225093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf2278-2a9b-4e7d-b6d6-0c12376efabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:10.226041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf2278-2a9b-4e7d-b6d6-0c12376efabe None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.226325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf2278-2a9b-4e7d-b6d6-0c12376efabe None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.226598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf2278-2a9b-4e7d-b6d6-0c12376efabe None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.227246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf2278-2a9b-4e7d-b6d6-0c12376efabe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:10.227623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf2278-2a9b-4e7d-b6d6-0c12376efabe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:10.230415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf2278-2a9b-4e7d-b6d6-0c12376efabe None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:10.237639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 532/1050] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:11:10 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:11:10.254738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 519/1051] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:10.259990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefea33-e24b-4029-865e-9c7f31b138ac None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:10.373659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefea33-e24b-4029-865e-9c7f31b138ac None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:10.396353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85531a3-668f-4586-bd41-9e5128d8550e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.396672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85531a3-668f-4586-bd41-9e5128d8550e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.396944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85531a3-668f-4586-bd41-9e5128d8550e None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.398756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 520/1052] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:11:10.404503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:10.405022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72839390-9126-4065-9886-e27858c7ef02 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.405412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72839390-9126-4065-9886-e27858c7ef02 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.405766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72839390-9126-4065-9886-e27858c7ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.420083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5614775-9e41-446d-81b7-798b69da0620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:10.421131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5614775-9e41-446d-81b7-798b69da0620 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.421131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5614775-9e41-446d-81b7-798b69da0620 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.421376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5614775-9e41-446d-81b7-798b69da0620 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.427994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-72839390-9126-4065-9886-e27858c7ef02 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:10.429092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5614775-9e41-446d-81b7-798b69da0620 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:10.429213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-72839390-9126-4065-9886-e27858c7ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:10.430809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5614775-9e41-446d-81b7-798b69da0620 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a5614775-9e41-446d-81b7-798b69da0620 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:11:10.442901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:10.443913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:10.444920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 533/1053] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Aug 19 15:11:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:10.452380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5614775-9e41-446d-81b7-798b69da0620 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:10.478289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:10.508719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 521/1054] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:10.560728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5614775-9e41-446d-81b7-798b69da0620 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:10.598179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2500b3-130a-4eb9-9c0b-eb5b14455003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:10.598793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2500b3-130a-4eb9-9c0b-eb5b14455003 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.599075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2500b3-130a-4eb9-9c0b-eb5b14455003 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.599351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2500b3-130a-4eb9-9c0b-eb5b14455003 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.600085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2500b3-130a-4eb9-9c0b-eb5b14455003 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:10.600401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2500b3-130a-4eb9-9c0b-eb5b14455003 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:10.601895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2500b3-130a-4eb9-9c0b-eb5b14455003 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:10.620212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 534/1055] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Aug 19 15:11:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:10.625698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72839390-9126-4065-9886-e27858c7ef02 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:10.734175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72839390-9126-4065-9886-e27858c7ef02 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:10.770834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f97d9-7caf-46e4-93c6-cd5fb56ddb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:10.771463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f97d9-7caf-46e4-93c6-cd5fb56ddb6e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.771739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f97d9-7caf-46e4-93c6-cd5fb56ddb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.772007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f97d9-7caf-46e4-93c6-cd5fb56ddb6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.775414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2500b3-130a-4eb9-9c0b-eb5b14455003 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:10.783661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f97d9-7caf-46e4-93c6-cd5fb56ddb6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:10.785163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f97d9-7caf-46e4-93c6-cd5fb56ddb6e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5f5f97d9-7caf-46e4-93c6-cd5fb56ddb6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:11:10.801887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:10.802901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:11:10.803841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:11:10.803841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:10.803841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:10.803841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 522/1056] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:10.809683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f97d9-7caf-46e4-93c6-cd5fb56ddb6e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:10.823754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:10.835061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:10.884074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2500b3-130a-4eb9-9c0b-eb5b14455003 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:10.917395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f97d9-7caf-46e4-93c6-cd5fb56ddb6e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:10.919199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286a2d-3960-40a5-9202-cf97b550b45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:10.919837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286a2d-3960-40a5-9202-cf97b550b45a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.920114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286a2d-3960-40a5-9202-cf97b550b45a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.920404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59286a2d-3960-40a5-9202-cf97b550b45a None admin] PATH_INFO: `/v3/services/image` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.921003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286a2d-3960-40a5-9202-cf97b550b45a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:10.922381 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59286a2d-3960-40a5-9202-cf97b550b45a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-59286a2d-3960-40a5-9202-cf97b550b45a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:10.930991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:10.932730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 15:11:10.933598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:10.936573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 535/1057] 10.222.0.75 () {56 vars in 1105 bytes} [Fri Aug 19 15:11:10 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:10.938455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286a2d-3960-40a5-9202-cf97b550b45a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:10.956927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ddaa0-1bdc-4c6b-a29e-1b24defbca74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:10.957535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ddaa0-1bdc-4c6b-a29e-1b24defbca74 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:10.957814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ddaa0-1bdc-4c6b-a29e-1b24defbca74 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:10.958083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ddaa0-1bdc-4c6b-a29e-1b24defbca74 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:10.958731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ddaa0-1bdc-4c6b-a29e-1b24defbca74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:10.959029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ddaa0-1bdc-4c6b-a29e-1b24defbca74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:10.960419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ddaa0-1bdc-4c6b-a29e-1b24defbca74 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:10.972161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 523/1058] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:10.978772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ddaa0-1bdc-4c6b-a29e-1b24defbca74 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:11.049944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59286a2d-3960-40a5-9202-cf97b550b45a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:11.059581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:11.065056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:11.086626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ddaa0-1bdc-4c6b-a29e-1b24defbca74 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:11.091700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295827-03ec-477b-9f5f-422a4c6cfdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:11.092309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295827-03ec-477b-9f5f-422a4c6cfdc9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:11.092627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295827-03ec-477b-9f5f-422a4c6cfdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:11.092901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295827-03ec-477b-9f5f-422a4c6cfdc9 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:11.093527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295827-03ec-477b-9f5f-422a4c6cfdc9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:11.093823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295827-03ec-477b-9f5f-422a4c6cfdc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:11.095202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295827-03ec-477b-9f5f-422a4c6cfdc9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:11.104433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 536/1059] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:11.106532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295827-03ec-477b-9f5f-422a4c6cfdc9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:11.128454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136f990-f50e-448b-909a-4678c622eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:11.129085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136f990-f50e-448b-909a-4678c622eba4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:11.129367 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136f990-f50e-448b-909a-4678c622eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:11.129640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136f990-f50e-448b-909a-4678c622eba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:11.137319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136f990-f50e-448b-909a-4678c622eba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:11.138716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136f990-f50e-448b-909a-4678c622eba4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6136f990-f50e-448b-909a-4678c622eba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:11:11.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:11.151707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:11.153133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:11:11.153133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:11:11.153133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:11:11.153133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:11:11.153133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:11:11.153133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:11:11.153133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:11:11.153133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:11.153133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 524/1060] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:11:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:11.157690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136f990-f50e-448b-909a-4678c622eba4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:11.220502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295827-03ec-477b-9f5f-422a4c6cfdc9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:11.255778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913bde0-b139-447f-82b4-de68a9c186a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:11.256399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913bde0-b139-447f-82b4-de68a9c186a9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:11.256691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913bde0-b139-447f-82b4-de68a9c186a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:11.257063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913bde0-b139-447f-82b4-de68a9c186a9 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:11.257593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913bde0-b139-447f-82b4-de68a9c186a9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:11.257896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913bde0-b139-447f-82b4-de68a9c186a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:11.259272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913bde0-b139-447f-82b4-de68a9c186a9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:11.269150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 537/1061] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:11.270088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913bde0-b139-447f-82b4-de68a9c186a9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:11.282822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136f990-f50e-448b-909a-4678c622eba4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:11.309004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:11.319940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5b863-b54f-4364-ae67-b531af3e1aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:11.320578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5b863-b54f-4364-ae67-b531af3e1aff None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:11.320858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5b863-b54f-4364-ae67-b531af3e1aff None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:11.321127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5b863-b54f-4364-ae67-b531af3e1aff None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:11.321927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5b863-b54f-4364-ae67-b531af3e1aff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:11.322236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5b863-b54f-4364-ae67-b531af3e1aff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:11.323765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5b863-b54f-4364-ae67-b531af3e1aff None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:11.336919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 525/1062] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:11:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:11.343626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5b863-b54f-4364-ae67-b531af3e1aff None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:11.346511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:11.383536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913bde0-b139-447f-82b4-de68a9c186a9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:11.422103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b1a2a-f056-4829-9790-f7606d73db96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:11.422678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b1a2a-f056-4829-9790-f7606d73db96 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:11.422957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b1a2a-f056-4829-9790-f7606d73db96 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:11.423227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b1a2a-f056-4829-9790-f7606d73db96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:11.423870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b1a2a-f056-4829-9790-f7606d73db96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:11.425279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b1a2a-f056-4829-9790-f7606d73db96 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:11.446269 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 538/1063] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:11:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:11:11.476275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5b863-b54f-4364-ae67-b531af3e1aff None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:11.521894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384358f-e576-4010-bbf1-ba03bed8cddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:11.522567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384358f-e576-4010-bbf1-ba03bed8cddd None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:11.522844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384358f-e576-4010-bbf1-ba03bed8cddd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:11.523116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384358f-e576-4010-bbf1-ba03bed8cddd None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/98a63d5b605a4ecfad2e305c491ae755/roles/68bd116b233640b7b7e07ef743a8be88` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:11.526239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384358f-e576-4010-bbf1-ba03bed8cddd None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=98a63d5b605a4ecfad2e305c491ae755, role_id=68bd116b233640b7b7e07ef743a8be88)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:11.528145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384358f-e576-4010-bbf1-ba03bed8cddd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:11.543725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 526/1064] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:11:11 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/98a63d5b605a4ecfad2e305c491ae755/roles/68bd116b233640b7b7e07ef743a8be88 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:11:12.245902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa227c84-90f7-47b7-9fc2-70f6a66e1675 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:12.246430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa227c84-90f7-47b7-9fc2-70f6a66e1675 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:12.246430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa227c84-90f7-47b7-9fc2-70f6a66e1675 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:12.247815 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 539/1065] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:12.252921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43143fb-5837-465b-820f-330b47654641 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:12.253215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43143fb-5837-465b-820f-330b47654641 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:12.253496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43143fb-5837-465b-820f-330b47654641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:12.273638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f43143fb-5837-465b-820f-330b47654641 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:12.273925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f43143fb-5837-465b-820f-330b47654641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:12.329913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:12.346429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 527/1066] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:12.614289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa227c84-90f7-47b7-9fc2-70f6a66e1675 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:12.722374 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:12.734957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa227c84-90f7-47b7-9fc2-70f6a66e1675 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:12.768721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683c39a-3e47-41f3-b4a1-fbd08c7ce239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:12.769343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683c39a-3e47-41f3-b4a1-fbd08c7ce239 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:12.769621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683c39a-3e47-41f3-b4a1-fbd08c7ce239 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:12.769894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683c39a-3e47-41f3-b4a1-fbd08c7ce239 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:12.785576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683c39a-3e47-41f3-b4a1-fbd08c7ce239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c683c39a-3e47-41f3-b4a1-fbd08c7ce239 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:11:12.794091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:12.795107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:12.797135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 540/1067] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:12.801796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43143fb-5837-465b-820f-330b47654641 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:12.904921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43143fb-5837-465b-820f-330b47654641 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:12.940009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d12413-4ed3-490e-8819-4b0df3cc4979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:12.940653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d12413-4ed3-490e-8819-4b0df3cc4979 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:12.940937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d12413-4ed3-490e-8819-4b0df3cc4979 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:12.941213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d12413-4ed3-490e-8819-4b0df3cc4979 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:12.941878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d12413-4ed3-490e-8819-4b0df3cc4979 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:12.942182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d12413-4ed3-490e-8819-4b0df3cc4979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:12.943560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d12413-4ed3-490e-8819-4b0df3cc4979 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:12.952077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 528/1068] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:11:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:12.959159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683c39a-3e47-41f3-b4a1-fbd08c7ce239 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:13.064587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683c39a-3e47-41f3-b4a1-fbd08c7ce239 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:13.099554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3846b03-a4a4-40e4-9343-bc5002d07851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:13.100191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3846b03-a4a4-40e4-9343-bc5002d07851 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:13.100498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3846b03-a4a4-40e4-9343-bc5002d07851 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:13.100772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3846b03-a4a4-40e4-9343-bc5002d07851 None admin] PATH_INFO: `/v3/users/05d4b238355c4af8b8c86ef984e04d65` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:13.114918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3846b03-a4a4-40e4-9343-bc5002d07851 None admin] RBAC: Authorizing `identity:get_user(user_id=05d4b238355c4af8b8c86ef984e04d65)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:13.116409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3846b03-a4a4-40e4-9343-bc5002d07851 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:13.118244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 541/1069] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:12 2022] GET /identity/v3/users/05d4b238355c4af8b8c86ef984e04d65 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:13.124311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d12413-4ed3-490e-8819-4b0df3cc4979 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:13.175673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:13.228937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d12413-4ed3-490e-8819-4b0df3cc4979 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:13.264708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af69b4-eaba-40d1-8873-bef415b78eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:13.265314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af69b4-eaba-40d1-8873-bef415b78eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:13.265592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af69b4-eaba-40d1-8873-bef415b78eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:13.265862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af69b4-eaba-40d1-8873-bef415b78eb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:13.335442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af69b4-eaba-40d1-8873-bef415b78eb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:13.336949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5af69b4-eaba-40d1-8873-bef415b78eb0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:13.387317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:13.427449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c549a-82bd-4679-a703-971a1a748e87 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:13.427947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c549a-82bd-4679-a703-971a1a748e87 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:13.427947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c549a-82bd-4679-a703-971a1a748e87 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:13.429523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 542/1070] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:13.435291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431242d-5a3b-4ff8-a943-be3981ed3922 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:13.435843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431242d-5a3b-4ff8-a943-be3981ed3922 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:13.436125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431242d-5a3b-4ff8-a943-be3981ed3922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:13.448356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 529/1071] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:11:13.449269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af69b4-eaba-40d1-8873-bef415b78eb0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:13.454242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5431242d-5a3b-4ff8-a943-be3981ed3922 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:13.455437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5431242d-5a3b-4ff8-a943-be3981ed3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:13.457539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:13.530880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 543/1072] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:13.532143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c514ea-39f9-426a-ba2a-59e8fad26e6a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:13.532458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c514ea-39f9-426a-ba2a-59e8fad26e6a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:13.532746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c514ea-39f9-426a-ba2a-59e8fad26e6a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:13.533976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 544/1073] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:13.541529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344847b-ad3f-48d7-9d83-524de910c2ab None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:13.542038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344847b-ad3f-48d7-9d83-524de910c2ab None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:13.542344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344847b-ad3f-48d7-9d83-524de910c2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:13.559126 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5af69b4-eaba-40d1-8873-bef415b78eb0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:13.564873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e344847b-ad3f-48d7-9d83-524de910c2ab None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:13.566705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e344847b-ad3f-48d7-9d83-524de910c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:13.614919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573b6cd-e8e0-4330-9bd5-290ef873d253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:13.615894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573b6cd-e8e0-4330-9bd5-290ef873d253 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:13.616517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573b6cd-e8e0-4330-9bd5-290ef873d253 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:13.616980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573b6cd-e8e0-4330-9bd5-290ef873d253 None admin] PATH_INFO: `/v3/projects/cf9d01ab001643c9b5d98773412b8b54` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:13.618810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573b6cd-e8e0-4330-9bd5-290ef873d253 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9d01ab001643c9b5d98773412b8b54)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:13.621679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573b6cd-e8e0-4330-9bd5-290ef873d253 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:13.628650 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 530/1074] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:11:13 2022] GET /identity/v3/projects/cf9d01ab001643c9b5d98773412b8b54 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:13.634154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573b6cd-e8e0-4330-9bd5-290ef873d253 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:13.657601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:13.674671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 545/1075] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:13.750983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573b6cd-e8e0-4330-9bd5-290ef873d253 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:13.786699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9afad-c68a-4b91-a87f-7b4cfcdb0fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:13.787302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9afad-c68a-4b91-a87f-7b4cfcdb0fdc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:13.787579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9afad-c68a-4b91-a87f-7b4cfcdb0fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:13.787848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9afad-c68a-4b91-a87f-7b4cfcdb0fdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:13.788603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9afad-c68a-4b91-a87f-7b4cfcdb0fdc None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=cf9d01ab001643c9b5d98773412b8b54, user.id=05d4b238355c4af8b8c86ef984e04d65) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:13.788912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9afad-c68a-4b91-a87f-7b4cfcdb0fdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:13.790348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9afad-c68a-4b91-a87f-7b4cfcdb0fdc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:13.796033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 531/1076] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:11:13 2022] GET /identity/v3/role_assignments?user.id=05d4b238355c4af8b8c86ef984e04d65&scope.project.id=cf9d01ab001643c9b5d98773412b8b54&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:13.796501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344847b-ad3f-48d7-9d83-524de910c2ab None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:13.874230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:13.914341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344847b-ad3f-48d7-9d83-524de910c2ab None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:13.950331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9afad-c68a-4b91-a87f-7b4cfcdb0fdc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:13.950918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdef5ea-8878-4819-89d7-aba1f3eed9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:13.951534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdef5ea-8878-4819-89d7-aba1f3eed9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:13.951817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdef5ea-8878-4819-89d7-aba1f3eed9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:13.952089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdef5ea-8878-4819-89d7-aba1f3eed9f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:14.027522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdef5ea-8878-4819-89d7-aba1f3eed9f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:14.029064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdef5ea-8878-4819-89d7-aba1f3eed9f3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:14.045851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:14.065907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9afad-c68a-4b91-a87f-7b4cfcdb0fdc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:14.100981 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:14.103939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daee887-c92d-45ad-88d8-53e4785673ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:14.105435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daee887-c92d-45ad-88d8-53e4785673ac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:14.106039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daee887-c92d-45ad-88d8-53e4785673ac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:14.106315 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daee887-c92d-45ad-88d8-53e4785673ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:14.117726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daee887-c92d-45ad-88d8-53e4785673ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0daee887-c92d-45ad-88d8-53e4785673ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:11:14.123846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:14.124829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:14.129369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 532/1077] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Aug 19 15:11:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:14.130803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daee887-c92d-45ad-88d8-53e4785673ac None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:14.145339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 546/1078] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 15:11:14.151923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdef5ea-8878-4819-89d7-aba1f3eed9f3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:14.240644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daee887-c92d-45ad-88d8-53e4785673ac None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:14.271685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdef5ea-8878-4819-89d7-aba1f3eed9f3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:14.278228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad7dc4-19c3-40ec-ab26-3b7658088e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:14.278774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad7dc4-19c3-40ec-ab26-3b7658088e60 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:14.279053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad7dc4-19c3-40ec-ab26-3b7658088e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:14.279325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad7dc4-19c3-40ec-ab26-3b7658088e60 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:14.280110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad7dc4-19c3-40ec-ab26-3b7658088e60 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:14.280433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad7dc4-19c3-40ec-ab26-3b7658088e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:14.281890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad7dc4-19c3-40ec-ab26-3b7658088e60 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:14.296281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 533/1079] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Aug 19 15:11:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:14.297442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad7dc4-19c3-40ec-ab26-3b7658088e60 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:14.302069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:14.312883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15a50a-bd65-43fb-96d0-0ea59838b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:14.313582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15a50a-bd65-43fb-96d0-0ea59838b178 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:14.313855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15a50a-bd65-43fb-96d0-0ea59838b178 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:14.314178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15a50a-bd65-43fb-96d0-0ea59838b178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:14.324890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15a50a-bd65-43fb-96d0-0ea59838b178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:14.326361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15a50a-bd65-43fb-96d0-0ea59838b178 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-da15a50a-bd65-43fb-96d0-0ea59838b178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:11:14.341453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:14.342627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:11:14.343707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:11:14.343707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:14.343707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:14.343707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 547/1080] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:14.348380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15a50a-bd65-43fb-96d0-0ea59838b178 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:14.371445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:14.414397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad7dc4-19c3-40ec-ab26-3b7658088e60 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:14.450927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68463014-34ba-477b-bc34-897977e91423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:14.451549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463014-34ba-477b-bc34-897977e91423 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:14.451824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463014-34ba-477b-bc34-897977e91423 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:14.452088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463014-34ba-477b-bc34-897977e91423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:14.455357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15a50a-bd65-43fb-96d0-0ea59838b178 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:14.464179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68463014-34ba-477b-bc34-897977e91423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:14.465709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68463014-34ba-477b-bc34-897977e91423 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-68463014-34ba-477b-bc34-897977e91423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:11:14.480798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:14.481938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:11:14.483043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:11:14.483043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:14.483043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:14.486794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 534/1081] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:14.487559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68463014-34ba-477b-bc34-897977e91423 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:14.496002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f6d68-9c04-40d9-9611-a4b3e6e60997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:14.496881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f6d68-9c04-40d9-9611-a4b3e6e60997 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:14.497366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f6d68-9c04-40d9-9611-a4b3e6e60997 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:14.497717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f6d68-9c04-40d9-9611-a4b3e6e60997 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:14.498365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f6d68-9c04-40d9-9611-a4b3e6e60997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:14.498672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f6d68-9c04-40d9-9611-a4b3e6e60997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:14.500064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f6d68-9c04-40d9-9611-a4b3e6e60997 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:14.512878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 548/1082] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:14.557767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:14.609754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68463014-34ba-477b-bc34-897977e91423 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:14.646712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fcde1-e7f6-4fc8-a28d-1694e15b1bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:14.647342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fcde1-e7f6-4fc8-a28d-1694e15b1bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:14.647627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fcde1-e7f6-4fc8-a28d-1694e15b1bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:14.647902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fcde1-e7f6-4fc8-a28d-1694e15b1bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:14.648577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fcde1-e7f6-4fc8-a28d-1694e15b1bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:14.648883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fcde1-e7f6-4fc8-a28d-1694e15b1bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:14.650836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fcde1-e7f6-4fc8-a28d-1694e15b1bf3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:14.664359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 535/1083] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:14.670307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f6d68-9c04-40d9-9611-a4b3e6e60997 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:14.763157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:14.786027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f6d68-9c04-40d9-9611-a4b3e6e60997 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:14.821641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bb8ca-ac42-466f-a44d-e8f1796558c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:14.822737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb8ca-ac42-466f-a44d-e8f1796558c3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:14.823020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb8ca-ac42-466f-a44d-e8f1796558c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:14.823288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb8ca-ac42-466f-a44d-e8f1796558c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:14.830765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bb8ca-ac42-466f-a44d-e8f1796558c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:14.832222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bb8ca-ac42-466f-a44d-e8f1796558c3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c73bb8ca-ac42-466f-a44d-e8f1796558c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:11:14.844714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:14.845818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:14.846774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 549/1084] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Aug 19 15:11:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:14.851579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fcde1-e7f6-4fc8-a28d-1694e15b1bf3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:14.954228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fcde1-e7f6-4fc8-a28d-1694e15b1bf3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:14.970551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:14.988507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19443795-0908-4dae-b9a7-9861bce3ae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:14.989101 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19443795-0908-4dae-b9a7-9861bce3ae7c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:14.989378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19443795-0908-4dae-b9a7-9861bce3ae7c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:14.989645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19443795-0908-4dae-b9a7-9861bce3ae7c None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:14.990366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19443795-0908-4dae-b9a7-9861bce3ae7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:14.990667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19443795-0908-4dae-b9a7-9861bce3ae7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:14.992095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19443795-0908-4dae-b9a7-9861bce3ae7c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:15.010776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 536/1085] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Aug 19 15:11:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:15.016461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bb8ca-ac42-466f-a44d-e8f1796558c3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:15.121325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bb8ca-ac42-466f-a44d-e8f1796558c3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:15.155095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7078b8-2041-4b63-9786-dd281292beb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:15.155835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7078b8-2041-4b63-9786-dd281292beb5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:15.156073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7078b8-2041-4b63-9786-dd281292beb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:15.156351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7078b8-2041-4b63-9786-dd281292beb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:15.168483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:15.224662 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7078b8-2041-4b63-9786-dd281292beb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:15.226134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7078b8-2041-4b63-9786-dd281292beb5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:15.324320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 550/1086] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:11:15.330050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19443795-0908-4dae-b9a7-9861bce3ae7c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:15.430305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19443795-0908-4dae-b9a7-9861bce3ae7c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:15.464369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19569ab-2e7c-49ad-8afb-1863f1577e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:15.465023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19569ab-2e7c-49ad-8afb-1863f1577e00 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:15.465305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19569ab-2e7c-49ad-8afb-1863f1577e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:15.465581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19569ab-2e7c-49ad-8afb-1863f1577e00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:15.475566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19569ab-2e7c-49ad-8afb-1863f1577e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:15.477048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19569ab-2e7c-49ad-8afb-1863f1577e00 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b19569ab-2e7c-49ad-8afb-1863f1577e00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:11:15.491692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:15.492671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:11:15.493607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:11:15.493607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:11:15.493607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:15.493607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 537/1087] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:15.498623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7078b8-2041-4b63-9786-dd281292beb5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:15.540483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:15.599103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7078b8-2041-4b63-9786-dd281292beb5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:15.632714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ec0bd-5eaf-4e8f-8c93-c0bd26b879e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:15.633313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ec0bd-5eaf-4e8f-8c93-c0bd26b879e5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:15.633586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ec0bd-5eaf-4e8f-8c93-c0bd26b879e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:15.633854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ec0bd-5eaf-4e8f-8c93-c0bd26b879e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:15.635131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ec0bd-5eaf-4e8f-8c93-c0bd26b879e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:15.635428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ec0bd-5eaf-4e8f-8c93-c0bd26b879e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:15.636823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ec0bd-5eaf-4e8f-8c93-c0bd26b879e5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:15.647702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 551/1088] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:15.653691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19569ab-2e7c-49ad-8afb-1863f1577e00 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:15.683910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:15.715936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58605ee-b759-47be-8228-b407dacd8179 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:15.716232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58605ee-b759-47be-8228-b407dacd8179 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:15.716542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58605ee-b759-47be-8228-b407dacd8179 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:15.717882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 552/1089] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:11:15.722720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70e5a6-5048-4330-97fe-90afa16f53ff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:15.723007 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70e5a6-5048-4330-97fe-90afa16f53ff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:15.723290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70e5a6-5048-4330-97fe-90afa16f53ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:15.740701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2e70e5a6-5048-4330-97fe-90afa16f53ff None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:15.741891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e70e5a6-5048-4330-97fe-90afa16f53ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:15.756589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19569ab-2e7c-49ad-8afb-1863f1577e00 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:15.790004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34007a8-fa56-4843-89d5-158aac7d7a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:15.790630 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34007a8-fa56-4843-89d5-158aac7d7a01 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:15.791526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34007a8-fa56-4843-89d5-158aac7d7a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:15.791889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34007a8-fa56-4843-89d5-158aac7d7a01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:15.798770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34007a8-fa56-4843-89d5-158aac7d7a01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:15.800217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34007a8-fa56-4843-89d5-158aac7d7a01 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:15.809613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 553/1090] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-c34007a8-fa56-4843-89d5-158aac7d7a01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:11:15.812640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:15.813952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:15.815067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:11:15.815067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:11:15.815067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:11:15.815067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:11:15.815067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:11:15.815067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:11:15.815067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:11:15.815067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:15.816643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 538/1091] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:11:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:15.819307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70e5a6-5048-4330-97fe-90afa16f53ff None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:15.849730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:15.921500 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70e5a6-5048-4330-97fe-90afa16f53ff None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:15.955677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829bde6-4619-4614-b3fa-b1c728496275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:15.956211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829bde6-4619-4614-b3fa-b1c728496275 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:15.956531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829bde6-4619-4614-b3fa-b1c728496275 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:15.956810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829bde6-4619-4614-b3fa-b1c728496275 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:15.957493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829bde6-4619-4614-b3fa-b1c728496275 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:15.957799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829bde6-4619-4614-b3fa-b1c728496275 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:15.959240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829bde6-4619-4614-b3fa-b1c728496275 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:15.970707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 554/1092] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:11:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:15.976711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34007a8-fa56-4843-89d5-158aac7d7a01 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:16.061705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:16.076804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34007a8-fa56-4843-89d5-158aac7d7a01 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:16.081902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829bde6-4619-4614-b3fa-b1c728496275 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:16.111948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03dd21e-ef41-4549-ad49-d449afd6df93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:16.112577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03dd21e-ef41-4549-ad49-d449afd6df93 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:16.112853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03dd21e-ef41-4549-ad49-d449afd6df93 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:16.113116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03dd21e-ef41-4549-ad49-d449afd6df93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:16.113834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03dd21e-ef41-4549-ad49-d449afd6df93 None admin] RBAC: Adding query filter params (role.id=68bd116b233640b7b7e07ef743a8be88, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=98a63d5b605a4ecfad2e305c491ae755) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:16.114133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03dd21e-ef41-4549-ad49-d449afd6df93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:16.115569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03dd21e-ef41-4549-ad49-d449afd6df93 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:16.122101 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 539/1093] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:11:15 2022] GET /identity/v3/role_assignments?user.id=98a63d5b605a4ecfad2e305c491ae755&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=68bd116b233640b7b7e07ef743a8be88 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:16.179786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:16.190657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829bde6-4619-4614-b3fa-b1c728496275 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:16.226335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18348ab9-c429-4a16-9784-310fd6255791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:16.226898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18348ab9-c429-4a16-9784-310fd6255791 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:16.227178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18348ab9-c429-4a16-9784-310fd6255791 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:16.227451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18348ab9-c429-4a16-9784-310fd6255791 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:16.239192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18348ab9-c429-4a16-9784-310fd6255791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-18348ab9-c429-4a16-9784-310fd6255791 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:11:16.245337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:16.246594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:16.247674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:16.248863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 555/1094] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:11:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:11:16.252850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03dd21e-ef41-4549-ad49-d449afd6df93 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:16.359616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03dd21e-ef41-4549-ad49-d449afd6df93 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:16.394543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73ce86-c887-42d9-a427-8dbee9cb17c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:16.395147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73ce86-c887-42d9-a427-8dbee9cb17c8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:16.395422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73ce86-c887-42d9-a427-8dbee9cb17c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:16.395688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73ce86-c887-42d9-a427-8dbee9cb17c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:16.396352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73ce86-c887-42d9-a427-8dbee9cb17c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:16.396668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73ce86-c887-42d9-a427-8dbee9cb17c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:16.398042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73ce86-c887-42d9-a427-8dbee9cb17c8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:16.407504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 540/1095] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:11:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:16.413571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18348ab9-c429-4a16-9784-310fd6255791 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:16.516984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18348ab9-c429-4a16-9784-310fd6255791 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:16.552467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b81205-4bf3-4c60-a711-4714691ea907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:16.553105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81205-4bf3-4c60-a711-4714691ea907 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:16.553384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81205-4bf3-4c60-a711-4714691ea907 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:16.553657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81205-4bf3-4c60-a711-4714691ea907 None admin] PATH_INFO: `/v3/users/05d4b238355c4af8b8c86ef984e04d65` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:16.554819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b81205-4bf3-4c60-a711-4714691ea907 None admin] RBAC: Authorizing `identity:get_user(user_id=05d4b238355c4af8b8c86ef984e04d65)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:16.556293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b81205-4bf3-4c60-a711-4714691ea907 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:16.558604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 556/1096] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:16 2022] GET /identity/v3/users/05d4b238355c4af8b8c86ef984e04d65 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:11:16.564260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73ce86-c887-42d9-a427-8dbee9cb17c8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:16.580769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:16.678940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73ce86-c887-42d9-a427-8dbee9cb17c8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:16.712322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd09a53-fb63-42b3-889e-04b264368a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:16.713746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd09a53-fb63-42b3-889e-04b264368a7e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:16.713866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd09a53-fb63-42b3-889e-04b264368a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:16.714051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd09a53-fb63-42b3-889e-04b264368a7e None admin] PATH_INFO: `/v3/projects/cf9d01ab001643c9b5d98773412b8b54` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:16.715177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd09a53-fb63-42b3-889e-04b264368a7e None admin] RBAC: Authorizing `identity:get_project(project_id=cf9d01ab001643c9b5d98773412b8b54)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:16.716739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd09a53-fb63-42b3-889e-04b264368a7e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:16.718436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 541/1097] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:11:16 2022] GET /identity/v3/projects/cf9d01ab001643c9b5d98773412b8b54 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:16.724763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b81205-4bf3-4c60-a711-4714691ea907 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:16.746165 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:16.829522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b81205-4bf3-4c60-a711-4714691ea907 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:16.865152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f85fca-347b-4234-bde9-3ac6bb8706c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:16.865855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f85fca-347b-4234-bde9-3ac6bb8706c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:16.866136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f85fca-347b-4234-bde9-3ac6bb8706c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:16.866408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f85fca-347b-4234-bde9-3ac6bb8706c4 None admin] PATH_INFO: `/v3/projects/cf9d01ab001643c9b5d98773412b8b54/users/05d4b238355c4af8b8c86ef984e04d65/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:16.868273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f85fca-347b-4234-bde9-3ac6bb8706c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf9d01ab001643c9b5d98773412b8b54, user_id=05d4b238355c4af8b8c86ef984e04d65, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:16.871162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f85fca-347b-4234-bde9-3ac6bb8706c4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:16.886189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 557/1098] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:11:16 2022] PUT /identity/v3/projects/cf9d01ab001643c9b5d98773412b8b54/users/05d4b238355c4af8b8c86ef984e04d65/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:11:18.771828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9871ff-c338-4bcd-84fb-d18341300aa3 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:18.772378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9871ff-c338-4bcd-84fb-d18341300aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:18.772378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9871ff-c338-4bcd-84fb-d18341300aa3 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:18.773727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 542/1099] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:18.779028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e175a92-6942-41df-a48d-7bdb0ef7ebab None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:18.779463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e175a92-6942-41df-a48d-7bdb0ef7ebab None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:18.779579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e175a92-6942-41df-a48d-7bdb0ef7ebab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:18.797229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7e175a92-6942-41df-a48d-7bdb0ef7ebab None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:18.798413 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e175a92-6942-41df-a48d-7bdb0ef7ebab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:18.863771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 558/1100] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:19.133761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9871ff-c338-4bcd-84fb-d18341300aa3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:19.216758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:19.236788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9871ff-c338-4bcd-84fb-d18341300aa3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:19.268981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269069f-428a-4d5f-859e-67df1da485fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:19.269594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269069f-428a-4d5f-859e-67df1da485fb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:19.269874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269069f-428a-4d5f-859e-67df1da485fb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:19.270145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269069f-428a-4d5f-859e-67df1da485fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:19.281169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269069f-428a-4d5f-859e-67df1da485fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4269069f-428a-4d5f-859e-67df1da485fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:11:19.285820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:19.286976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:19.288237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 543/1101] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:11:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:19.294111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e175a92-6942-41df-a48d-7bdb0ef7ebab None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:19.304336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:19.391442 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e175a92-6942-41df-a48d-7bdb0ef7ebab None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:19.425228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debda9c9-419c-48c6-b079-1c6cf678d07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:19.425819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debda9c9-419c-48c6-b079-1c6cf678d07f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:19.426098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debda9c9-419c-48c6-b079-1c6cf678d07f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:19.426370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debda9c9-419c-48c6-b079-1c6cf678d07f None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:19.427026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debda9c9-419c-48c6-b079-1c6cf678d07f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:19.427326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debda9c9-419c-48c6-b079-1c6cf678d07f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:19.428737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debda9c9-419c-48c6-b079-1c6cf678d07f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:19.436847 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 559/1102] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:11:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:19.442807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269069f-428a-4d5f-859e-67df1da485fb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:19.540925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269069f-428a-4d5f-859e-67df1da485fb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:19.577496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e202f57-72b1-4316-82d1-be4f08781050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:19.578116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202f57-72b1-4316-82d1-be4f08781050 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:19.578397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202f57-72b1-4316-82d1-be4f08781050 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:19.578671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e202f57-72b1-4316-82d1-be4f08781050 None admin] PATH_INFO: `/v3/users/05d4b238355c4af8b8c86ef984e04d65` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:19.579782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e202f57-72b1-4316-82d1-be4f08781050 None admin] RBAC: Authorizing `identity:get_user(user_id=05d4b238355c4af8b8c86ef984e04d65)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:19.581255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e202f57-72b1-4316-82d1-be4f08781050 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:19.582822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 544/1103] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:19 2022] GET /identity/v3/users/05d4b238355c4af8b8c86ef984e04d65 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:19.588930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debda9c9-419c-48c6-b079-1c6cf678d07f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:19.656714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:19.688440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debda9c9-419c-48c6-b079-1c6cf678d07f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:19.722531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763660d-eb32-4b31-9cd3-cf2fdd0b03bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:19.723128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763660d-eb32-4b31-9cd3-cf2fdd0b03bc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:19.723409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763660d-eb32-4b31-9cd3-cf2fdd0b03bc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:19.723683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763660d-eb32-4b31-9cd3-cf2fdd0b03bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:19.790007 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763660d-eb32-4b31-9cd3-cf2fdd0b03bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:19.791975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763660d-eb32-4b31-9cd3-cf2fdd0b03bc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:19.856964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:19.890780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 560/1104] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:11:19.896767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e202f57-72b1-4316-82d1-be4f08781050 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:19.908612 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:20.000475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e202f57-72b1-4316-82d1-be4f08781050 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:20.033074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb086780-807a-4cd3-a0d5-7c7786bde83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:20.033688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086780-807a-4cd3-a0d5-7c7786bde83e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:20.033969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086780-807a-4cd3-a0d5-7c7786bde83e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:20.034241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086780-807a-4cd3-a0d5-7c7786bde83e None admin] PATH_INFO: `/v3/projects/cf9d01ab001643c9b5d98773412b8b54` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:20.035337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb086780-807a-4cd3-a0d5-7c7786bde83e None admin] RBAC: Authorizing `identity:get_project(project_id=cf9d01ab001643c9b5d98773412b8b54)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:20.036796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb086780-807a-4cd3-a0d5-7c7786bde83e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:20.038433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 545/1105] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:11:19 2022] GET /identity/v3/projects/cf9d01ab001643c9b5d98773412b8b54 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:20.046080 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763660d-eb32-4b31-9cd3-cf2fdd0b03bc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:20.145544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763660d-eb32-4b31-9cd3-cf2fdd0b03bc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:20.179640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187ec81-7e89-401c-8e3f-fd83351c4637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:20.180169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187ec81-7e89-401c-8e3f-fd83351c4637 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:20.180477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187ec81-7e89-401c-8e3f-fd83351c4637 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:20.180754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187ec81-7e89-401c-8e3f-fd83351c4637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:20.181500 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187ec81-7e89-401c-8e3f-fd83351c4637 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=cf9d01ab001643c9b5d98773412b8b54, user.id=05d4b238355c4af8b8c86ef984e04d65) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:20.181806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187ec81-7e89-401c-8e3f-fd83351c4637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:20.183243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187ec81-7e89-401c-8e3f-fd83351c4637 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:20.189003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 561/1106] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:11:20 2022] GET /identity/v3/role_assignments?user.id=05d4b238355c4af8b8c86ef984e04d65&scope.project.id=cf9d01ab001643c9b5d98773412b8b54&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:22.095432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d187c79-42e8-4931-adab-58e4bf9cad23 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:22.095985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d187c79-42e8-4931-adab-58e4bf9cad23 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:22.095985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d187c79-42e8-4931-adab-58e4bf9cad23 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:22.097333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 546/1107] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:22.102377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594698d-f37f-4fcc-95ab-389aba16cd12 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:22.102785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594698d-f37f-4fcc-95ab-389aba16cd12 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:22.102918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594698d-f37f-4fcc-95ab-389aba16cd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:22.121802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0594698d-f37f-4fcc-95ab-389aba16cd12 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:22.122975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0594698d-f37f-4fcc-95ab-389aba16cd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:22.149150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:22.191686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 562/1108] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:22.473047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d187c79-42e8-4931-adab-58e4bf9cad23 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:22.550600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:22.573616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d187c79-42e8-4931-adab-58e4bf9cad23 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:22.606377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a2aa6-103a-4f58-ac88-a43905d1c42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:22.606983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2aa6-103a-4f58-ac88-a43905d1c42d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:22.607256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2aa6-103a-4f58-ac88-a43905d1c42d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:22.607521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2aa6-103a-4f58-ac88-a43905d1c42d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:22.608631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a2aa6-103a-4f58-ac88-a43905d1c42d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:22.610053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a2aa6-103a-4f58-ac88-a43905d1c42d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:22.611581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 547/1109] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:22.617258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594698d-f37f-4fcc-95ab-389aba16cd12 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:22.719997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594698d-f37f-4fcc-95ab-389aba16cd12 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:22.753572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954fa46-2b6b-426a-94e5-54bc62ff1fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:22.754106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954fa46-2b6b-426a-94e5-54bc62ff1fb2 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:22.754391 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954fa46-2b6b-426a-94e5-54bc62ff1fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:22.754663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954fa46-2b6b-426a-94e5-54bc62ff1fb2 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:22.755315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954fa46-2b6b-426a-94e5-54bc62ff1fb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:22.756790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954fa46-2b6b-426a-94e5-54bc62ff1fb2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:22.778696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 563/1110] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:11:22 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:11:24.781203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67017679-e36f-4edd-9e02-ec282048cdc0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:24.781748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67017679-e36f-4edd-9e02-ec282048cdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:24.781748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67017679-e36f-4edd-9e02-ec282048cdc0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:24.783075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 548/1111] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:24.788279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc78c7-c8d0-438c-9b89-742f5ea39bb7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:24.788727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc78c7-c8d0-438c-9b89-742f5ea39bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:24.788857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc78c7-c8d0-438c-9b89-742f5ea39bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:24.807469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a9cc78c7-c8d0-438c-9b89-742f5ea39bb7 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:24.808670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a9cc78c7-c8d0-438c-9b89-742f5ea39bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:24.822619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:24.875900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 564/1112] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:25.145035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67017679-e36f-4edd-9e02-ec282048cdc0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:25.247764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67017679-e36f-4edd-9e02-ec282048cdc0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:25.282273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f2b66-b434-4511-b07b-51df1921829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:25.282887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f2b66-b434-4511-b07b-51df1921829e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:25.283166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f2b66-b434-4511-b07b-51df1921829e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:25.283437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f2b66-b434-4511-b07b-51df1921829e None admin] PATH_INFO: `/v3/roles/048908af57db471987eb7f2ec21e0304` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:25.284851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f2b66-b434-4511-b07b-51df1921829e None admin] RBAC: Authorizing `identity:get_role(role_id=048908af57db471987eb7f2ec21e0304)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:25.286221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f2b66-b434-4511-b07b-51df1921829e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:25.287543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 549/1113] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:25 2022] GET /identity/v3/roles/048908af57db471987eb7f2ec21e0304 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:25.293533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc78c7-c8d0-438c-9b89-742f5ea39bb7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:25.390963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc78c7-c8d0-438c-9b89-742f5ea39bb7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:25.424437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a798566-1e88-431f-94e3-950b5172c4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:25.424999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a798566-1e88-431f-94e3-950b5172c4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:25.425280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a798566-1e88-431f-94e3-950b5172c4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:25.425557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a798566-1e88-431f-94e3-950b5172c4a3 None admin] PATH_INFO: `/v3/users/deca98f9cbb74e4b9bff9a1d9366d3b7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:25.438795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a798566-1e88-431f-94e3-950b5172c4a3 None admin] RBAC: Authorizing `identity:get_user(user_id=deca98f9cbb74e4b9bff9a1d9366d3b7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:25.440267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a798566-1e88-431f-94e3-950b5172c4a3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:25.441894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 565/1114] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:25 2022] GET /identity/v3/users/deca98f9cbb74e4b9bff9a1d9366d3b7 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:25.448375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f2b66-b434-4511-b07b-51df1921829e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:25.463433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:25.545542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f2b66-b434-4511-b07b-51df1921829e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:25.578605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ff784-99ec-464f-acd6-4c6223834670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:25.579195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ff784-99ec-464f-acd6-4c6223834670 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:25.579474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ff784-99ec-464f-acd6-4c6223834670 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:25.579746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ff784-99ec-464f-acd6-4c6223834670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:25.650970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ff784-99ec-464f-acd6-4c6223834670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:25.652449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ff784-99ec-464f-acd6-4c6223834670 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:25.665787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:25.751833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 550/1115] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 15:11:25.757701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a798566-1e88-431f-94e3-950b5172c4a3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:25.768924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:25.857265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a798566-1e88-431f-94e3-950b5172c4a3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:25.891554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17ea0f-6163-475d-ae3b-f66d47446fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:25.892166 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17ea0f-6163-475d-ae3b-f66d47446fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:25.892475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17ea0f-6163-475d-ae3b-f66d47446fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:25.892772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17ea0f-6163-475d-ae3b-f66d47446fa5 None admin] PATH_INFO: `/v3/projects/cf9d01ab001643c9b5d98773412b8b54` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:25.893859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17ea0f-6163-475d-ae3b-f66d47446fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9d01ab001643c9b5d98773412b8b54)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:25.895303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17ea0f-6163-475d-ae3b-f66d47446fa5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:25.896958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 566/1116] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:11:25 2022] GET /identity/v3/projects/cf9d01ab001643c9b5d98773412b8b54 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:25.903679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ff784-99ec-464f-acd6-4c6223834670 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:26.008228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ff784-99ec-464f-acd6-4c6223834670 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:26.020939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:26.042738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecd54e-329e-4623-8da8-2cdebbb062ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:26.043334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecd54e-329e-4623-8da8-2cdebbb062ca None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:26.043613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecd54e-329e-4623-8da8-2cdebbb062ca None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:26.043910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecd54e-329e-4623-8da8-2cdebbb062ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:26.044633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecd54e-329e-4623-8da8-2cdebbb062ca None admin] RBAC: Adding query filter params (role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=cf9d01ab001643c9b5d98773412b8b54, user.id=deca98f9cbb74e4b9bff9a1d9366d3b7) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:26.044941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecd54e-329e-4623-8da8-2cdebbb062ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:26.046380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecd54e-329e-4623-8da8-2cdebbb062ca None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:26.051616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 551/1117] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:11:25 2022] GET /identity/v3/role_assignments?user.id=deca98f9cbb74e4b9bff9a1d9366d3b7&scope.project.id=cf9d01ab001643c9b5d98773412b8b54&role.id=048908af57db471987eb7f2ec21e0304 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:28.035136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cec552-0056-4942-90d6-df99c4c78d72 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:28.035693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cec552-0056-4942-90d6-df99c4c78d72 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:28.035693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cec552-0056-4942-90d6-df99c4c78d72 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:28.037051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 567/1118] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:28.042752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b078e-03f1-4ef7-835f-030eb1357761 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:28.043170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b078e-03f1-4ef7-835f-030eb1357761 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:28.043299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b078e-03f1-4ef7-835f-030eb1357761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:28.061543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-891b078e-03f1-4ef7-835f-030eb1357761 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:28.062731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-891b078e-03f1-4ef7-835f-030eb1357761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:28.130108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 552/1119] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:28.397466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cec552-0056-4942-90d6-df99c4c78d72 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:28.476355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:28.500670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cec552-0056-4942-90d6-df99c4c78d72 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:28.534095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08be2b-2ecd-47f0-ad7f-65f770a24637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:28.534706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08be2b-2ecd-47f0-ad7f-65f770a24637 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:28.534981 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08be2b-2ecd-47f0-ad7f-65f770a24637 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:28.535252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08be2b-2ecd-47f0-ad7f-65f770a24637 None admin] PATH_INFO: `/v3/roles/048908af57db471987eb7f2ec21e0304` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:28.536651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08be2b-2ecd-47f0-ad7f-65f770a24637 None admin] RBAC: Authorizing `identity:get_role(role_id=048908af57db471987eb7f2ec21e0304)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:28.538045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08be2b-2ecd-47f0-ad7f-65f770a24637 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:28.539335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 568/1120] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:28 2022] GET /identity/v3/roles/048908af57db471987eb7f2ec21e0304 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:28.545975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b078e-03f1-4ef7-835f-030eb1357761 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:28.643887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b078e-03f1-4ef7-835f-030eb1357761 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:28.652231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:28.680034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491e79-d615-4b29-ae8a-b07c54db6bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:28.680694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491e79-d615-4b29-ae8a-b07c54db6bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:28.680971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491e79-d615-4b29-ae8a-b07c54db6bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:28.681571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491e79-d615-4b29-ae8a-b07c54db6bc5 None admin] PATH_INFO: `/v3/users/deca98f9cbb74e4b9bff9a1d9366d3b7` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:28.682689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62491e79-d615-4b29-ae8a-b07c54db6bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=deca98f9cbb74e4b9bff9a1d9366d3b7)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:28.684125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62491e79-d615-4b29-ae8a-b07c54db6bc5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:28.685730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 553/1121] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:28 2022] GET /identity/v3/users/deca98f9cbb74e4b9bff9a1d9366d3b7 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:28.691439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08be2b-2ecd-47f0-ad7f-65f770a24637 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:28.789900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08be2b-2ecd-47f0-ad7f-65f770a24637 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:28.824016 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b639d3-37d5-4c92-8cc0-8aad99170297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:28.824711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b639d3-37d5-4c92-8cc0-8aad99170297 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:28.824994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b639d3-37d5-4c92-8cc0-8aad99170297 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:28.825270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b639d3-37d5-4c92-8cc0-8aad99170297 None admin] PATH_INFO: `/v3/projects/cf9d01ab001643c9b5d98773412b8b54` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:28.826385 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b639d3-37d5-4c92-8cc0-8aad99170297 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9d01ab001643c9b5d98773412b8b54)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:28.827951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b639d3-37d5-4c92-8cc0-8aad99170297 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:28.829625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 569/1122] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:11:28 2022] GET /identity/v3/projects/cf9d01ab001643c9b5d98773412b8b54 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:28.835700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491e79-d615-4b29-ae8a-b07c54db6bc5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:28.936238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491e79-d615-4b29-ae8a-b07c54db6bc5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:28.969517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f611c8-a604-42c5-a59d-3bc11e10dc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:28.970158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f611c8-a604-42c5-a59d-3bc11e10dc02 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:28.970439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f611c8-a604-42c5-a59d-3bc11e10dc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:28.970712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f611c8-a604-42c5-a59d-3bc11e10dc02 None admin] PATH_INFO: `/v3/projects/cf9d01ab001643c9b5d98773412b8b54/users/deca98f9cbb74e4b9bff9a1d9366d3b7/roles/048908af57db471987eb7f2ec21e0304` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:28.972851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f611c8-a604-42c5-a59d-3bc11e10dc02 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf9d01ab001643c9b5d98773412b8b54, user_id=deca98f9cbb74e4b9bff9a1d9366d3b7, role_id=048908af57db471987eb7f2ec21e0304)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:28.974373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f611c8-a604-42c5-a59d-3bc11e10dc02 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:28.986941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 554/1123] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:11:28 2022] PUT /identity/v3/projects/cf9d01ab001643c9b5d98773412b8b54/users/deca98f9cbb74e4b9bff9a1d9366d3b7/roles/048908af57db471987eb7f2ec21e0304 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:11:30.912216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38948497-49ec-4f8b-ba96-7a3a5dc68d67 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:30.912800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38948497-49ec-4f8b-ba96-7a3a5dc68d67 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:30.912800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38948497-49ec-4f8b-ba96-7a3a5dc68d67 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:30.914117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 570/1124] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:30.919407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6e427-daed-40e5-95b9-050cdfbd8f77 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:30.919845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6e427-daed-40e5-95b9-050cdfbd8f77 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:30.919950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6e427-daed-40e5-95b9-050cdfbd8f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:30.937989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b5d6e427-daed-40e5-95b9-050cdfbd8f77 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:30.939078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b5d6e427-daed-40e5-95b9-050cdfbd8f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:30.964969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:31.004793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 555/1125] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:31.278521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38948497-49ec-4f8b-ba96-7a3a5dc68d67 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:31.293930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:31.378907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38948497-49ec-4f8b-ba96-7a3a5dc68d67 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:31.412538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a91ac4-fe6f-40b1-b971-300f77df3eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:31.413152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a91ac4-fe6f-40b1-b971-300f77df3eed None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:31.413424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a91ac4-fe6f-40b1-b971-300f77df3eed None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:31.413690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a91ac4-fe6f-40b1-b971-300f77df3eed None admin] PATH_INFO: `/v3/roles/048908af57db471987eb7f2ec21e0304` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:31.415057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a91ac4-fe6f-40b1-b971-300f77df3eed None admin] RBAC: Authorizing `identity:get_role(role_id=048908af57db471987eb7f2ec21e0304)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:31.417354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a91ac4-fe6f-40b1-b971-300f77df3eed None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:31.418647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 571/1126] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:31 2022] GET /identity/v3/roles/048908af57db471987eb7f2ec21e0304 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:31.424504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6e427-daed-40e5-95b9-050cdfbd8f77 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:31.519562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6e427-daed-40e5-95b9-050cdfbd8f77 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:31.553003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebf05d-23ba-4bbb-baa0-9009683f2551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:31.553553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebf05d-23ba-4bbb-baa0-9009683f2551 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:31.553833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebf05d-23ba-4bbb-baa0-9009683f2551 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:31.554106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebf05d-23ba-4bbb-baa0-9009683f2551 None admin] PATH_INFO: `/v3/users/deca98f9cbb74e4b9bff9a1d9366d3b7` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:31.555203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebf05d-23ba-4bbb-baa0-9009683f2551 None admin] RBAC: Authorizing `identity:get_user(user_id=deca98f9cbb74e4b9bff9a1d9366d3b7)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:31.556813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebf05d-23ba-4bbb-baa0-9009683f2551 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:31.558369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 556/1127] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:31 2022] GET /identity/v3/users/deca98f9cbb74e4b9bff9a1d9366d3b7 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:31.564160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a91ac4-fe6f-40b1-b971-300f77df3eed None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:31.642549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:31.661905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a91ac4-fe6f-40b1-b971-300f77df3eed None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:31.696060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2165923-7f6e-4143-90a1-3f086b6c07de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:31.696729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2165923-7f6e-4143-90a1-3f086b6c07de None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:31.697142 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2165923-7f6e-4143-90a1-3f086b6c07de None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:31.697419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2165923-7f6e-4143-90a1-3f086b6c07de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:31.763191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2165923-7f6e-4143-90a1-3f086b6c07de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:31.764803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2165923-7f6e-4143-90a1-3f086b6c07de None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:31.842508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:31.861864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 572/1128] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:11:31.867890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebf05d-23ba-4bbb-baa0-9009683f2551 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:31.898276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:31.966843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebf05d-23ba-4bbb-baa0-9009683f2551 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:31.999242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a9796-333d-4bdc-ada7-7926d66fd626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:31.999855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9796-333d-4bdc-ada7-7926d66fd626 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:32.000135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9796-333d-4bdc-ada7-7926d66fd626 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:32.000427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9796-333d-4bdc-ada7-7926d66fd626 None admin] PATH_INFO: `/v3/projects/cf9d01ab001643c9b5d98773412b8b54` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:32.001533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a9796-333d-4bdc-ada7-7926d66fd626 None admin] RBAC: Authorizing `identity:get_project(project_id=cf9d01ab001643c9b5d98773412b8b54)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:32.002974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a9796-333d-4bdc-ada7-7926d66fd626 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:32.004617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 557/1129] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:11:31 2022] GET /identity/v3/projects/cf9d01ab001643c9b5d98773412b8b54 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:32.012249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2165923-7f6e-4143-90a1-3f086b6c07de None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:32.110297 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2165923-7f6e-4143-90a1-3f086b6c07de None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:32.143379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b56b8-ddf3-4b33-beae-876c9b0fa495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:32.143965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b56b8-ddf3-4b33-beae-876c9b0fa495 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:32.144239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b56b8-ddf3-4b33-beae-876c9b0fa495 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:32.144533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b56b8-ddf3-4b33-beae-876c9b0fa495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:32.145247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b56b8-ddf3-4b33-beae-876c9b0fa495 None admin] RBAC: Adding query filter params (role.id=048908af57db471987eb7f2ec21e0304, scope.project.id=cf9d01ab001643c9b5d98773412b8b54, user.id=deca98f9cbb74e4b9bff9a1d9366d3b7) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:32.145550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b56b8-ddf3-4b33-beae-876c9b0fa495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:32.146974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b56b8-ddf3-4b33-beae-876c9b0fa495 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:32.152609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 573/1130] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:11:32 2022] GET /identity/v3/role_assignments?user.id=deca98f9cbb74e4b9bff9a1d9366d3b7&scope.project.id=cf9d01ab001643c9b5d98773412b8b54&role.id=048908af57db471987eb7f2ec21e0304 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:34.074419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fac1e-f02d-489b-bda8-a1c534c93e66 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:34.074975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fac1e-f02d-489b-bda8-a1c534c93e66 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:34.074975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0fac1e-f02d-489b-bda8-a1c534c93e66 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:34.076292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 558/1131] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:34.081579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9ab56-b2c2-4caf-bb25-8d3e03f57ff7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:34.081984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9ab56-b2c2-4caf-bb25-8d3e03f57ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:34.082131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9ab56-b2c2-4caf-bb25-8d3e03f57ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:34.099657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e0c9ab56-b2c2-4caf-bb25-8d3e03f57ff7 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:34.100869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e0c9ab56-b2c2-4caf-bb25-8d3e03f57ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:34.140638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:34.169034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 574/1132] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:34.439990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fac1e-f02d-489b-bda8-a1c534c93e66 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:34.536260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:34.541445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0fac1e-f02d-489b-bda8-a1c534c93e66 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:34.574423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf428f-f532-4530-89b1-9bbeb3494e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:34.575037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf428f-f532-4530-89b1-9bbeb3494e0d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:34.575558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf428f-f532-4530-89b1-9bbeb3494e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:34.575820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf428f-f532-4530-89b1-9bbeb3494e0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:34.576946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf428f-f532-4530-89b1-9bbeb3494e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:34.578474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf428f-f532-4530-89b1-9bbeb3494e0d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:34.580027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 559/1133] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:34.585912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9ab56-b2c2-4caf-bb25-8d3e03f57ff7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:34.684237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9ab56-b2c2-4caf-bb25-8d3e03f57ff7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:34.717969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd22303-e1ea-4d6c-aa17-16a10ae5f888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:34.718502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22303-e1ea-4d6c-aa17-16a10ae5f888 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:34.718783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22303-e1ea-4d6c-aa17-16a10ae5f888 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:34.719057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22303-e1ea-4d6c-aa17-16a10ae5f888 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:34.719711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd22303-e1ea-4d6c-aa17-16a10ae5f888 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:34.721184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd22303-e1ea-4d6c-aa17-16a10ae5f888 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:34.748053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 575/1134] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:11:34 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:11:36.683734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160f4f2-819e-48da-b03e-1db8b22bd8c8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:36.684194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160f4f2-819e-48da-b03e-1db8b22bd8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:36.684283 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160f4f2-819e-48da-b03e-1db8b22bd8c8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:36.685663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 560/1135] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:36.690893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481b899-5a68-47c3-b6b8-bf42b0ecee72 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:36.691284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481b899-5a68-47c3-b6b8-bf42b0ecee72 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:36.691439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481b899-5a68-47c3-b6b8-bf42b0ecee72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:36.709177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7481b899-5a68-47c3-b6b8-bf42b0ecee72 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:36.710364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7481b899-5a68-47c3-b6b8-bf42b0ecee72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:36.731825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:36.778437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 576/1136] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:37.048235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160f4f2-819e-48da-b03e-1db8b22bd8c8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:37.146307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160f4f2-819e-48da-b03e-1db8b22bd8c8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:37.180209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febb8bd-bd76-4bb5-b172-3afef3f62814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:37.180844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febb8bd-bd76-4bb5-b172-3afef3f62814 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:37.182011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febb8bd-bd76-4bb5-b172-3afef3f62814 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:37.182011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febb8bd-bd76-4bb5-b172-3afef3f62814 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:37.183196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febb8bd-bd76-4bb5-b172-3afef3f62814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:37.184659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febb8bd-bd76-4bb5-b172-3afef3f62814 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:37.186192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 561/1137] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:11:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:37.192014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481b899-5a68-47c3-b6b8-bf42b0ecee72 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:37.292322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481b899-5a68-47c3-b6b8-bf42b0ecee72 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:37.325940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0c6fd-9405-45ef-b722-3da9c1bf4c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:37.326474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c6fd-9405-45ef-b722-3da9c1bf4c49 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:37.326752 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c6fd-9405-45ef-b722-3da9c1bf4c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:37.327025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c6fd-9405-45ef-b722-3da9c1bf4c49 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:37.327674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0c6fd-9405-45ef-b722-3da9c1bf4c49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:37.329140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0c6fd-9405-45ef-b722-3da9c1bf4c49 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:37.350303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 577/1138] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:11:37 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:11:39.280664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152a23-7553-47b8-9e23-db39c2dc5966 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:39.281194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152a23-7553-47b8-9e23-db39c2dc5966 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:39.281194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152a23-7553-47b8-9e23-db39c2dc5966 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:39.282535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 562/1139] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:39.287666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5fac6-5382-4ea2-8518-2aad867495ae None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:39.288070 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5fac6-5382-4ea2-8518-2aad867495ae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:39.288200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5fac6-5382-4ea2-8518-2aad867495ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:39.306860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dbf5fac6-5382-4ea2-8518-2aad867495ae None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:39.308188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dbf5fac6-5382-4ea2-8518-2aad867495ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:39.315553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:39.377069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 578/1140] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:39.649931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152a23-7553-47b8-9e23-db39c2dc5966 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:39.687586 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:39.753357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152a23-7553-47b8-9e23-db39c2dc5966 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:39.786680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb6ae4-e8aa-49ce-8f37-ca90280db4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:39.787296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb6ae4-e8aa-49ce-8f37-ca90280db4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:39.787574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb6ae4-e8aa-49ce-8f37-ca90280db4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:39.787843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb6ae4-e8aa-49ce-8f37-ca90280db4d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:39.797683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb6ae4-e8aa-49ce-8f37-ca90280db4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-cabb6ae4-e8aa-49ce-8f37-ca90280db4d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:11:39.802263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:39.803392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:39.804607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 563/1141] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:11:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:39.810201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5fac6-5382-4ea2-8518-2aad867495ae None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:39.909207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5fac6-5382-4ea2-8518-2aad867495ae None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:39.943977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d4265-9151-48f0-9821-96f555e19948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:39.944600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d4265-9151-48f0-9821-96f555e19948 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:39.944874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d4265-9151-48f0-9821-96f555e19948 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:39.945162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d4265-9151-48f0-9821-96f555e19948 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:39.945817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d4265-9151-48f0-9821-96f555e19948 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:39.946112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d4265-9151-48f0-9821-96f555e19948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:39.948075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d4265-9151-48f0-9821-96f555e19948 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:39.956109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 579/1142] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:11:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:39.961898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb6ae4-e8aa-49ce-8f37-ca90280db4d1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:40.055798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:40.062527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb6ae4-e8aa-49ce-8f37-ca90280db4d1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:40.096464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c578eb2-7219-415f-afd4-808eb867f9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:40.097076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c578eb2-7219-415f-afd4-808eb867f9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:40.097353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c578eb2-7219-415f-afd4-808eb867f9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:40.097622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c578eb2-7219-415f-afd4-808eb867f9f6 None admin] PATH_INFO: `/v3/users/2913acd355d541bd80dd124e44f8827f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:40.111560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c578eb2-7219-415f-afd4-808eb867f9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=2913acd355d541bd80dd124e44f8827f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:40.113028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c578eb2-7219-415f-afd4-808eb867f9f6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:40.114606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 564/1143] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:39 2022] GET /identity/v3/users/2913acd355d541bd80dd124e44f8827f => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:40.121191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d4265-9151-48f0-9821-96f555e19948 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:40.124945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:40.218575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d4265-9151-48f0-9821-96f555e19948 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:40.252379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d30b3-9a54-4f94-b552-e6e323948adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:40.252999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d30b3-9a54-4f94-b552-e6e323948adf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:40.253280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d30b3-9a54-4f94-b552-e6e323948adf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:40.253557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d30b3-9a54-4f94-b552-e6e323948adf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:40.318940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d30b3-9a54-4f94-b552-e6e323948adf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:40.320416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d30b3-9a54-4f94-b552-e6e323948adf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:40.323216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:40.418042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 580/1144] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:11:40.424103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c578eb2-7219-415f-afd4-808eb867f9f6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:40.519704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c578eb2-7219-415f-afd4-808eb867f9f6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:40.553769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee60cd-28d8-4c5a-979f-262e140dfcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:40.554381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee60cd-28d8-4c5a-979f-262e140dfcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:40.554660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee60cd-28d8-4c5a-979f-262e140dfcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:40.554931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee60cd-28d8-4c5a-979f-262e140dfcf8 None admin] PATH_INFO: `/v3/projects/041646eb431a47908e2069c65fe36438` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:40.556054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee60cd-28d8-4c5a-979f-262e140dfcf8 None admin] RBAC: Authorizing `identity:get_project(project_id=041646eb431a47908e2069c65fe36438)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:40.557514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee60cd-28d8-4c5a-979f-262e140dfcf8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:40.559119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 565/1145] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:11:40 2022] GET /identity/v3/projects/041646eb431a47908e2069c65fe36438 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:40.565246 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d30b3-9a54-4f94-b552-e6e323948adf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:40.665177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d30b3-9a54-4f94-b552-e6e323948adf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:40.668362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:40.700266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482527d-fb3f-46c0-bf43-6d0a0105dd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:40.700899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482527d-fb3f-46c0-bf43-6d0a0105dd07 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:40.701179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482527d-fb3f-46c0-bf43-6d0a0105dd07 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:40.701450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482527d-fb3f-46c0-bf43-6d0a0105dd07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:40.702165 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482527d-fb3f-46c0-bf43-6d0a0105dd07 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=041646eb431a47908e2069c65fe36438, user.id=2913acd355d541bd80dd124e44f8827f) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:40.702488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482527d-fb3f-46c0-bf43-6d0a0105dd07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:40.703904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482527d-fb3f-46c0-bf43-6d0a0105dd07 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:40.709163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 581/1146] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:11:40 2022] GET /identity/v3/role_assignments?user.id=2913acd355d541bd80dd124e44f8827f&scope.project.id=041646eb431a47908e2069c65fe36438&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:42.582576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4a286-e447-4421-a8fc-7f019eeeae15 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:42.583130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4a286-e447-4421-a8fc-7f019eeeae15 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:42.583130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4a286-e447-4421-a8fc-7f019eeeae15 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:42.584476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 566/1147] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:42.590451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07985a04-6537-4b8a-af96-0e15cf06ddad None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:42.590923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07985a04-6537-4b8a-af96-0e15cf06ddad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:42.591051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07985a04-6537-4b8a-af96-0e15cf06ddad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:42.608785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-07985a04-6537-4b8a-af96-0e15cf06ddad None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:42.610004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-07985a04-6537-4b8a-af96-0e15cf06ddad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:42.675624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 582/1148] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:42.945383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4a286-e447-4421-a8fc-7f019eeeae15 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:42.970128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:43.048837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4a286-e447-4421-a8fc-7f019eeeae15 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:43.082145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98664b5a-dd4d-4882-b1cd-f93486474b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:43.082747 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98664b5a-dd4d-4882-b1cd-f93486474b02 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:43.083023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98664b5a-dd4d-4882-b1cd-f93486474b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:43.083292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98664b5a-dd4d-4882-b1cd-f93486474b02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:43.093355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98664b5a-dd4d-4882-b1cd-f93486474b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-98664b5a-dd4d-4882-b1cd-f93486474b02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:11:43.097912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:43.099038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:43.099894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 567/1149] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:11:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:43.105739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07985a04-6537-4b8a-af96-0e15cf06ddad None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:43.200798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:43.204778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07985a04-6537-4b8a-af96-0e15cf06ddad None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:43.239041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72081-a883-437f-997a-f52e063a15cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:43.239642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72081-a883-437f-997a-f52e063a15cd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:43.239915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72081-a883-437f-997a-f52e063a15cd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:43.240183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef72081-a883-437f-997a-f52e063a15cd None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:43.240867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72081-a883-437f-997a-f52e063a15cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:43.241167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72081-a883-437f-997a-f52e063a15cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:43.242921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef72081-a883-437f-997a-f52e063a15cd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:43.251518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 583/1150] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:11:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:43.257280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98664b5a-dd4d-4882-b1cd-f93486474b02 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:43.338746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:43.358233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98664b5a-dd4d-4882-b1cd-f93486474b02 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:43.391147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8a351-3d8c-4445-bc01-54005d85dba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:43.391762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8a351-3d8c-4445-bc01-54005d85dba0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:43.392039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8a351-3d8c-4445-bc01-54005d85dba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:43.392309 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8a351-3d8c-4445-bc01-54005d85dba0 None admin] PATH_INFO: `/v3/users/2913acd355d541bd80dd124e44f8827f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:43.393446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8a351-3d8c-4445-bc01-54005d85dba0 None admin] RBAC: Authorizing `identity:get_user(user_id=2913acd355d541bd80dd124e44f8827f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:43.394884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8a351-3d8c-4445-bc01-54005d85dba0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:43.396480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 568/1151] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:43 2022] GET /identity/v3/users/2913acd355d541bd80dd124e44f8827f => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:43.402834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72081-a883-437f-997a-f52e063a15cd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:43.501741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef72081-a883-437f-997a-f52e063a15cd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:43.534651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b957a5-ff8b-40fb-b27f-a1fc7e91ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:43.535273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b957a5-ff8b-40fb-b27f-a1fc7e91ff40 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:43.535554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b957a5-ff8b-40fb-b27f-a1fc7e91ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:43.535831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b957a5-ff8b-40fb-b27f-a1fc7e91ff40 None admin] PATH_INFO: `/v3/projects/041646eb431a47908e2069c65fe36438` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:43.536963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b957a5-ff8b-40fb-b27f-a1fc7e91ff40 None admin] RBAC: Authorizing `identity:get_project(project_id=041646eb431a47908e2069c65fe36438)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:43.538407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b957a5-ff8b-40fb-b27f-a1fc7e91ff40 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:43.540188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 584/1152] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:11:43 2022] GET /identity/v3/projects/041646eb431a47908e2069c65fe36438 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:43.546652 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8a351-3d8c-4445-bc01-54005d85dba0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:43.643270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8a351-3d8c-4445-bc01-54005d85dba0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:43.680299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1cf92-7f86-46b8-b703-3b2b9271ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:43.681011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1cf92-7f86-46b8-b703-3b2b9271ddac None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:43.681359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1cf92-7f86-46b8-b703-3b2b9271ddac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:43.681689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1cf92-7f86-46b8-b703-3b2b9271ddac None admin] PATH_INFO: `/v3/projects/041646eb431a47908e2069c65fe36438/users/2913acd355d541bd80dd124e44f8827f/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:43.683619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1cf92-7f86-46b8-b703-3b2b9271ddac None admin] RBAC: Authorizing `identity:create_grant(project_id=041646eb431a47908e2069c65fe36438, user_id=2913acd355d541bd80dd124e44f8827f, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:43.685096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1cf92-7f86-46b8-b703-3b2b9271ddac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:43.699002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 569/1153] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:11:43 2022] PUT /identity/v3/projects/041646eb431a47908e2069c65fe36438/users/2913acd355d541bd80dd124e44f8827f/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:11:45.585632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904a677-6d29-4ea5-8eaa-3f864e2743a0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:45.586187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904a677-6d29-4ea5-8eaa-3f864e2743a0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:45.586187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904a677-6d29-4ea5-8eaa-3f864e2743a0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:45.588150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 585/1154] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:45.593343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b34b6e-1aa6-4c20-80d6-69c1be7f2b92 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:45.593746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b34b6e-1aa6-4c20-80d6-69c1be7f2b92 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:45.593885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b34b6e-1aa6-4c20-80d6-69c1be7f2b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:45.611289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-12b34b6e-1aa6-4c20-80d6-69c1be7f2b92 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:45.612479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-12b34b6e-1aa6-4c20-80d6-69c1be7f2b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:45.615120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:45.679373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 570/1155] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:45.948488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904a677-6d29-4ea5-8eaa-3f864e2743a0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:45.976902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:46.049531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904a677-6d29-4ea5-8eaa-3f864e2743a0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:46.082089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d303e-1b09-4577-8d45-d93ae51a0abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:46.082706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d303e-1b09-4577-8d45-d93ae51a0abf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:46.082985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d303e-1b09-4577-8d45-d93ae51a0abf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:46.083254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d303e-1b09-4577-8d45-d93ae51a0abf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:46.093859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d303e-1b09-4577-8d45-d93ae51a0abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c47d303e-1b09-4577-8d45-d93ae51a0abf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:11:46.098471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:46.099572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:11:46.100551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 586/1156] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:11:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:46.106220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b34b6e-1aa6-4c20-80d6-69c1be7f2b92 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:46.202687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b34b6e-1aa6-4c20-80d6-69c1be7f2b92 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:46.235477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166a83-68da-403e-8f9c-4ff63366ec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:46.236068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166a83-68da-403e-8f9c-4ff63366ec95 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:46.236346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166a83-68da-403e-8f9c-4ff63366ec95 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:46.236649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166a83-68da-403e-8f9c-4ff63366ec95 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:46.237320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166a83-68da-403e-8f9c-4ff63366ec95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:46.237624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166a83-68da-403e-8f9c-4ff63366ec95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:46.238999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166a83-68da-403e-8f9c-4ff63366ec95 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:46.247519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 571/1157] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:11:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:46.253368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d303e-1b09-4577-8d45-d93ae51a0abf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:46.340932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:46.352601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d303e-1b09-4577-8d45-d93ae51a0abf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:46.385809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb6326-7887-450c-8b3b-ea41ff1c12e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:46.386409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb6326-7887-450c-8b3b-ea41ff1c12e1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:46.386690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb6326-7887-450c-8b3b-ea41ff1c12e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:46.386958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb6326-7887-450c-8b3b-ea41ff1c12e1 None admin] PATH_INFO: `/v3/users/2913acd355d541bd80dd124e44f8827f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:46.388047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb6326-7887-450c-8b3b-ea41ff1c12e1 None admin] RBAC: Authorizing `identity:get_user(user_id=2913acd355d541bd80dd124e44f8827f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:46.389573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb6326-7887-450c-8b3b-ea41ff1c12e1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:46.391116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 587/1158] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:11:46 2022] GET /identity/v3/users/2913acd355d541bd80dd124e44f8827f => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:46.397051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166a83-68da-403e-8f9c-4ff63366ec95 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:46.399828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:46.496870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166a83-68da-403e-8f9c-4ff63366ec95 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:46.529279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda00370-1742-4b76-b62c-8faf140fd666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:46.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00370-1742-4b76-b62c-8faf140fd666 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:46.530144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00370-1742-4b76-b62c-8faf140fd666 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:46.530416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00370-1742-4b76-b62c-8faf140fd666 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:46.597976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda00370-1742-4b76-b62c-8faf140fd666 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:46.599429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda00370-1742-4b76-b62c-8faf140fd666 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:46.601208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:46.697505 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 572/1159] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 15:11:46.703384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb6326-7887-450c-8b3b-ea41ff1c12e1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:46.802040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb6326-7887-450c-8b3b-ea41ff1c12e1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:46.834579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1628c5f-7029-4f71-b8d8-a398d691500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:46.835193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628c5f-7029-4f71-b8d8-a398d691500b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:46.835473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628c5f-7029-4f71-b8d8-a398d691500b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:46.835750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628c5f-7029-4f71-b8d8-a398d691500b None admin] PATH_INFO: `/v3/projects/041646eb431a47908e2069c65fe36438` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:46.836871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1628c5f-7029-4f71-b8d8-a398d691500b None admin] RBAC: Authorizing `identity:get_project(project_id=041646eb431a47908e2069c65fe36438)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:46.838900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1628c5f-7029-4f71-b8d8-a398d691500b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:46.840511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 588/1160] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:11:46 2022] GET /identity/v3/projects/041646eb431a47908e2069c65fe36438 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:46.846757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda00370-1742-4b76-b62c-8faf140fd666 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:46.948985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda00370-1742-4b76-b62c-8faf140fd666 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:46.950459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:46.983938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda1e95-28e3-4995-8a02-f155aaa1b8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:46.984553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda1e95-28e3-4995-8a02-f155aaa1b8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:46.984830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda1e95-28e3-4995-8a02-f155aaa1b8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:46.985099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda1e95-28e3-4995-8a02-f155aaa1b8e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:46.985813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda1e95-28e3-4995-8a02-f155aaa1b8e4 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=041646eb431a47908e2069c65fe36438, user.id=2913acd355d541bd80dd124e44f8827f) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:46.986117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda1e95-28e3-4995-8a02-f155aaa1b8e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:46.987542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda1e95-28e3-4995-8a02-f155aaa1b8e4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:46.993118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 573/1161] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:11:46 2022] GET /identity/v3/role_assignments?user.id=2913acd355d541bd80dd124e44f8827f&scope.project.id=041646eb431a47908e2069c65fe36438&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:48.907938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ed056-7c96-43cc-a4e2-a645b1e018de None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:48.908506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ed056-7c96-43cc-a4e2-a645b1e018de None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:48.908506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ed056-7c96-43cc-a4e2-a645b1e018de None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:48.909848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 589/1162] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:48.915021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c48d25-a302-4d48-8991-517323877d32 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:48.915471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c48d25-a302-4d48-8991-517323877d32 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:48.916265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c48d25-a302-4d48-8991-517323877d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:48.934071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f1c48d25-a302-4d48-8991-517323877d32 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:48.935197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f1c48d25-a302-4d48-8991-517323877d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:49.000688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 574/1163] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:49.278489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ed056-7c96-43cc-a4e2-a645b1e018de None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:49.307542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:49.383532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ed056-7c96-43cc-a4e2-a645b1e018de None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:49.416628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490c32d-6749-44f3-bdae-bb2a1133afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:49.417212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490c32d-6749-44f3-bdae-bb2a1133afbc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:49.417491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490c32d-6749-44f3-bdae-bb2a1133afbc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:49.417762 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490c32d-6749-44f3-bdae-bb2a1133afbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:49.481999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490c32d-6749-44f3-bdae-bb2a1133afbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:49.483445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490c32d-6749-44f3-bdae-bb2a1133afbc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:49.506702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:49.579869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 590/1164] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:11:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:11:49.585674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c48d25-a302-4d48-8991-517323877d32 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:49.677318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:49.683938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c48d25-a302-4d48-8991-517323877d32 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:49.717700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f696aaf-4a94-48a3-bfd8-ac7b002d178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:49.718315 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f696aaf-4a94-48a3-bfd8-ac7b002d178c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:49.718593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f696aaf-4a94-48a3-bfd8-ac7b002d178c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:49.718860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f696aaf-4a94-48a3-bfd8-ac7b002d178c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:49.725418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f696aaf-4a94-48a3-bfd8-ac7b002d178c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:49.726872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f696aaf-4a94-48a3-bfd8-ac7b002d178c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9f696aaf-4a94-48a3-bfd8-ac7b002d178c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:11:49.738760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:11:49.739979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:11:49.741055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 575/1165] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:11:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:11:49.746722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490c32d-6749-44f3-bdae-bb2a1133afbc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:49.843804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490c32d-6749-44f3-bdae-bb2a1133afbc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:49.866752 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:49.878210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a6f11-d629-463b-82a5-dc32d88eddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:49.878808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a6f11-d629-463b-82a5-dc32d88eddb9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:49.879092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a6f11-d629-463b-82a5-dc32d88eddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:49.879361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a6f11-d629-463b-82a5-dc32d88eddb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:49.879991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a6f11-d629-463b-82a5-dc32d88eddb9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:11:49.880288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a6f11-d629-463b-82a5-dc32d88eddb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:49.881695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a6f11-d629-463b-82a5-dc32d88eddb9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:49.888617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 591/1166] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Aug 19 15:11:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:51.826595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be0134-d033-448d-9bf8-f13230c8ce0a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:51.827178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be0134-d033-448d-9bf8-f13230c8ce0a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:51.827178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be0134-d033-448d-9bf8-f13230c8ce0a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:51.828515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 576/1167] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:51.833548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c48a-26f0-4855-8247-159d6af8fe49 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:51.833989 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c48a-26f0-4855-8247-159d6af8fe49 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:51.834088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c48a-26f0-4855-8247-159d6af8fe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:51.852323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ebe9c48a-26f0-4855-8247-159d6af8fe49 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:51.854402 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ebe9c48a-26f0-4855-8247-159d6af8fe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:51.919270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 592/1168] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:52.190294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be0134-d033-448d-9bf8-f13230c8ce0a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:52.288770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be0134-d033-448d-9bf8-f13230c8ce0a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:52.321482 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30005188-dad2-45ed-941c-5f536bc7ad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:52.322240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30005188-dad2-45ed-941c-5f536bc7ad5e None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:52.322519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30005188-dad2-45ed-941c-5f536bc7ad5e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:52.322790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30005188-dad2-45ed-941c-5f536bc7ad5e None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:52.323448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30005188-dad2-45ed-941c-5f536bc7ad5e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:52.324853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30005188-dad2-45ed-941c-5f536bc7ad5e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:52.340457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 577/1169] 10.222.0.75 () {60 vars in 1138 bytes} [Fri Aug 19 15:11:52 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:11:54.253247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e747f-a78a-4322-92f8-0302032edf6c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:54.253806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e747f-a78a-4322-92f8-0302032edf6c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:54.253806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e747f-a78a-4322-92f8-0302032edf6c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:54.255114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 593/1170] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:54.260824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ff69b-6fb8-449d-b1e5-1f87ff51d736 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:54.261263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ff69b-6fb8-449d-b1e5-1f87ff51d736 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:54.261492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ff69b-6fb8-449d-b1e5-1f87ff51d736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:54.280401 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-465ff69b-6fb8-449d-b1e5-1f87ff51d736 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:54.281594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-465ff69b-6fb8-449d-b1e5-1f87ff51d736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:54.292916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:54.348769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 578/1171] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:54.626564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e747f-a78a-4322-92f8-0302032edf6c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:54.731034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e747f-a78a-4322-92f8-0302032edf6c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:54.748572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:54.766912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbd890-60e3-40e1-8cfd-cc7de5b0c58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:54.767626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbd890-60e3-40e1-8cfd-cc7de5b0c58b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:54.767901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbd890-60e3-40e1-8cfd-cc7de5b0c58b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:54.768171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbd890-60e3-40e1-8cfd-cc7de5b0c58b None admin] PATH_INFO: `/v3/domains/7a9f13e846614c9c8794562ff9e7bfef` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:54.779436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbd890-60e3-40e1-8cfd-cc7de5b0c58b None admin] RBAC: Authorizing `identity:get_domain(domain_id=7a9f13e846614c9c8794562ff9e7bfef)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:54.780912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbd890-60e3-40e1-8cfd-cc7de5b0c58b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:54.782465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 594/1172] 10.222.0.75 () {56 vars in 1183 bytes} [Fri Aug 19 15:11:54 2022] GET /identity/v3/domains/7a9f13e846614c9c8794562ff9e7bfef => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:54.788487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ff69b-6fb8-449d-b1e5-1f87ff51d736 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:54.886019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ff69b-6fb8-449d-b1e5-1f87ff51d736 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:54.919589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080a608-93db-49a1-b223-c57fdaff7bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:54.920183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080a608-93db-49a1-b223-c57fdaff7bb7 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:54.920495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080a608-93db-49a1-b223-c57fdaff7bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:54.920765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080a608-93db-49a1-b223-c57fdaff7bb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:54.921459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080a608-93db-49a1-b223-c57fdaff7bb7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:54.922988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080a608-93db-49a1-b223-c57fdaff7bb7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:54.944897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:54.952855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 579/1173] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:11:54 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:11:56.900670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef85703-320f-4064-bc34-60797733b7f8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:56.901302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef85703-320f-4064-bc34-60797733b7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:56.901302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef85703-320f-4064-bc34-60797733b7f8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:56.902551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 595/1174] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:56.908077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08834d8-9ad3-46be-b91a-2e6a2a9717d5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:56.908552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08834d8-9ad3-46be-b91a-2e6a2a9717d5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:56.908663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08834d8-9ad3-46be-b91a-2e6a2a9717d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:56.926970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a08834d8-9ad3-46be-b91a-2e6a2a9717d5 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:56.928082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a08834d8-9ad3-46be-b91a-2e6a2a9717d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:56.994980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 580/1175] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:11:57.265961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef85703-320f-4064-bc34-60797733b7f8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:57.367746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef85703-320f-4064-bc34-60797733b7f8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:57.401600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45b650-d6ec-4aa7-9660-5ad303a2f713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:57.402219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45b650-d6ec-4aa7-9660-5ad303a2f713 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:57.402501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45b650-d6ec-4aa7-9660-5ad303a2f713 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:57.402774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45b650-d6ec-4aa7-9660-5ad303a2f713 None admin] PATH_INFO: `/v3/domains/7a9f13e846614c9c8794562ff9e7bfef` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:57.404138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45b650-d6ec-4aa7-9660-5ad303a2f713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7a9f13e846614c9c8794562ff9e7bfef)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:57.405616 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45b650-d6ec-4aa7-9660-5ad303a2f713 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:57.407174 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 596/1176] 10.222.0.75 () {56 vars in 1183 bytes} [Fri Aug 19 15:11:57 2022] GET /identity/v3/domains/7a9f13e846614c9c8794562ff9e7bfef => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:11:57.413096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08834d8-9ad3-46be-b91a-2e6a2a9717d5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:57.510150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08834d8-9ad3-46be-b91a-2e6a2a9717d5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:11:57.535245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:57.543754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad275b45-e5c5-4b10-bc9e-7ce06cfdff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:11:57.544353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad275b45-e5c5-4b10-bc9e-7ce06cfdff28 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:57.544662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad275b45-e5c5-4b10-bc9e-7ce06cfdff28 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:57.544932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad275b45-e5c5-4b10-bc9e-7ce06cfdff28 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:57.545562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad275b45-e5c5-4b10-bc9e-7ce06cfdff28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:11:57.546992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad275b45-e5c5-4b10-bc9e-7ce06cfdff28 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:11:57.567660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 581/1177] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:11:57 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:11:59.510072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62e944-3d59-45c5-bdab-7396a832fccb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:59.510645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62e944-3d59-45c5-bdab-7396a832fccb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:59.510645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62e944-3d59-45c5-bdab-7396a832fccb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:59.511941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 597/1178] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:11:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:11:59.517140 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0059121-219c-47c6-b20c-649f531e152c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:11:59.517527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0059121-219c-47c6-b20c-649f531e152c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:11:59.517689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0059121-219c-47c6-b20c-649f531e152c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:11:59.536237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c0059121-219c-47c6-b20c-649f531e152c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:11:59.537438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c0059121-219c-47c6-b20c-649f531e152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:11:59.602144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 582/1179] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:11:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:11:59.879913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62e944-3d59-45c5-bdab-7396a832fccb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:11:59.929771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:11:59.985212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62e944-3d59-45c5-bdab-7396a832fccb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:00.025248 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ab724-632e-4ef5-9783-23f2d25a1903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:00.025879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab724-632e-4ef5-9783-23f2d25a1903 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:00.026161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab724-632e-4ef5-9783-23f2d25a1903 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:00.026435 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab724-632e-4ef5-9783-23f2d25a1903 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:00.037153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ab724-632e-4ef5-9783-23f2d25a1903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a78ab724-632e-4ef5-9783-23f2d25a1903 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:12:00.041832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:12:00.043005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:12:00.044104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 598/1180] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:12:00.049050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0059121-219c-47c6-b20c-649f531e152c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:00.145876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0059121-219c-47c6-b20c-649f531e152c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:00.160022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:00.180478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211c398-9936-4ae3-9844-6b8f67c36dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:00.181636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211c398-9936-4ae3-9844-6b8f67c36dca None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:00.181910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211c398-9936-4ae3-9844-6b8f67c36dca None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:00.182186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211c398-9936-4ae3-9844-6b8f67c36dca None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:00.182850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211c398-9936-4ae3-9844-6b8f67c36dca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:12:00.183152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211c398-9936-4ae3-9844-6b8f67c36dca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:00.184561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211c398-9936-4ae3-9844-6b8f67c36dca None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:00.192535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 583/1181] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:12:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:00.199520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ab724-632e-4ef5-9783-23f2d25a1903 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:00.297905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ab724-632e-4ef5-9783-23f2d25a1903 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:00.304521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:00.331817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f46a9-f507-4d7e-98a4-a140cec56713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:00.332472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f46a9-f507-4d7e-98a4-a140cec56713 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:00.332759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f46a9-f507-4d7e-98a4-a140cec56713 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:00.333040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f46a9-f507-4d7e-98a4-a140cec56713 None admin] PATH_INFO: `/v3/users/ed037216ad65437388bca1c9c4e28d3b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:00.350916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f46a9-f507-4d7e-98a4-a140cec56713 None admin] RBAC: Authorizing `identity:get_user(user_id=ed037216ad65437388bca1c9c4e28d3b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:00.352373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f46a9-f507-4d7e-98a4-a140cec56713 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:00.354571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 599/1182] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:12:00 2022] GET /identity/v3/users/ed037216ad65437388bca1c9c4e28d3b => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:00.360570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211c398-9936-4ae3-9844-6b8f67c36dca None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:00.458344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211c398-9936-4ae3-9844-6b8f67c36dca None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:00.490688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbc218-9ea2-47ae-8a0e-b77d0fee9ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:00.491272 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbc218-9ea2-47ae-8a0e-b77d0fee9ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:00.491549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbc218-9ea2-47ae-8a0e-b77d0fee9ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:00.491814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbc218-9ea2-47ae-8a0e-b77d0fee9ea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:00.535916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:00.559076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbc218-9ea2-47ae-8a0e-b77d0fee9ea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:00.560568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbc218-9ea2-47ae-8a0e-b77d0fee9ea2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:00.654940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 584/1183] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:12:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:12:00.660866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f46a9-f507-4d7e-98a4-a140cec56713 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:00.758905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f46a9-f507-4d7e-98a4-a140cec56713 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:00.791907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1991b24-ee5f-4276-9bfb-ab208219ab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:00.792541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1991b24-ee5f-4276-9bfb-ab208219ab89 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:00.792818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1991b24-ee5f-4276-9bfb-ab208219ab89 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:00.793084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1991b24-ee5f-4276-9bfb-ab208219ab89 None admin] PATH_INFO: `/v3/projects/6b678825ee3d405fb7aea9381dc8d9fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:00.794176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1991b24-ee5f-4276-9bfb-ab208219ab89 None admin] RBAC: Authorizing `identity:get_project(project_id=6b678825ee3d405fb7aea9381dc8d9fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:00.795598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1991b24-ee5f-4276-9bfb-ab208219ab89 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:00.797228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 600/1184] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:12:00 2022] GET /identity/v3/projects/6b678825ee3d405fb7aea9381dc8d9fd => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:00.803286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbc218-9ea2-47ae-8a0e-b77d0fee9ea2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:00.878500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:00.900995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbc218-9ea2-47ae-8a0e-b77d0fee9ea2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:00.934127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1b20d-dae0-48b0-963e-1913596cd854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:00.934718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1b20d-dae0-48b0-963e-1913596cd854 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:00.934998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1b20d-dae0-48b0-963e-1913596cd854 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:00.935266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1b20d-dae0-48b0-963e-1913596cd854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:00.935987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1b20d-dae0-48b0-963e-1913596cd854 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=6b678825ee3d405fb7aea9381dc8d9fd, user.id=ed037216ad65437388bca1c9c4e28d3b) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:12:00.936287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1b20d-dae0-48b0-963e-1913596cd854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:00.937741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1b20d-dae0-48b0-963e-1913596cd854 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:00.943087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 585/1185] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:12:00 2022] GET /identity/v3/role_assignments?user.id=ed037216ad65437388bca1c9c4e28d3b&scope.project.id=6b678825ee3d405fb7aea9381dc8d9fd&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:02.805318 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bf4d3-b089-4c1d-a4a0-0b3aac6b4cb2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:02.805877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bf4d3-b089-4c1d-a4a0-0b3aac6b4cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:02.805877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bf4d3-b089-4c1d-a4a0-0b3aac6b4cb2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:02.807566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 601/1186] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:12:02.812748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef00ac0-b25c-4e3b-bb4c-83166075ec4b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:02.813154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef00ac0-b25c-4e3b-bb4c-83166075ec4b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:02.813294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef00ac0-b25c-4e3b-bb4c-83166075ec4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:02.831378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5ef00ac0-b25c-4e3b-bb4c-83166075ec4b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:12:02.832594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5ef00ac0-b25c-4e3b-bb4c-83166075ec4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:12:02.898188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 586/1187] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:12:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:12:03.168801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bf4d3-b089-4c1d-a4a0-0b3aac6b4cb2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:03.200723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:03.270705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bf4d3-b089-4c1d-a4a0-0b3aac6b4cb2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:03.305229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc183f-a65f-46e0-9ff8-94409b248f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:03.305850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc183f-a65f-46e0-9ff8-94409b248f98 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:03.307156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc183f-a65f-46e0-9ff8-94409b248f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:03.307156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc183f-a65f-46e0-9ff8-94409b248f98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:03.318866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc183f-a65f-46e0-9ff8-94409b248f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-68cc183f-a65f-46e0-9ff8-94409b248f98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:12:03.323731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:12:03.325050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:12:03.326295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 602/1188] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:12:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:12:03.331072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef00ac0-b25c-4e3b-bb4c-83166075ec4b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:03.399136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:03.430091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef00ac0-b25c-4e3b-bb4c-83166075ec4b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:03.464599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0f4f7-662c-40fe-95e2-2e940de5f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:03.465186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0f4f7-662c-40fe-95e2-2e940de5f012 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:03.465461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0f4f7-662c-40fe-95e2-2e940de5f012 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:03.465728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0f4f7-662c-40fe-95e2-2e940de5f012 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:03.466382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0f4f7-662c-40fe-95e2-2e940de5f012 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:12:03.466679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0f4f7-662c-40fe-95e2-2e940de5f012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:03.468202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0f4f7-662c-40fe-95e2-2e940de5f012 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:03.476120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 587/1189] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:12:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:03.482578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc183f-a65f-46e0-9ff8-94409b248f98 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:03.573260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:03.581976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc183f-a65f-46e0-9ff8-94409b248f98 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:03.615884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687a7b-6331-4874-948a-c6cf5fd8e418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:03.616525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687a7b-6331-4874-948a-c6cf5fd8e418 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:03.616805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687a7b-6331-4874-948a-c6cf5fd8e418 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:03.617078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687a7b-6331-4874-948a-c6cf5fd8e418 None admin] PATH_INFO: `/v3/users/ed037216ad65437388bca1c9c4e28d3b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:03.618190 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f687a7b-6331-4874-948a-c6cf5fd8e418 None admin] RBAC: Authorizing `identity:get_user(user_id=ed037216ad65437388bca1c9c4e28d3b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:03.620334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f687a7b-6331-4874-948a-c6cf5fd8e418 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:03.621909 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 603/1190] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:12:03 2022] GET /identity/v3/users/ed037216ad65437388bca1c9c4e28d3b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:03.627694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0f4f7-662c-40fe-95e2-2e940de5f012 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:03.726161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0f4f7-662c-40fe-95e2-2e940de5f012 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:03.754328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:03.759489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c9a5d-6186-41b5-9985-e37ba027e8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:03.760190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c9a5d-6186-41b5-9985-e37ba027e8fc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:03.760564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c9a5d-6186-41b5-9985-e37ba027e8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:03.760840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c9a5d-6186-41b5-9985-e37ba027e8fc None admin] PATH_INFO: `/v3/projects/6b678825ee3d405fb7aea9381dc8d9fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:03.761979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c9a5d-6186-41b5-9985-e37ba027e8fc None admin] RBAC: Authorizing `identity:get_project(project_id=6b678825ee3d405fb7aea9381dc8d9fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:03.763422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c9a5d-6186-41b5-9985-e37ba027e8fc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:03.765144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 588/1191] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:12:03 2022] GET /identity/v3/projects/6b678825ee3d405fb7aea9381dc8d9fd => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:03.771219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687a7b-6331-4874-948a-c6cf5fd8e418 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:03.868814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687a7b-6331-4874-948a-c6cf5fd8e418 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:03.901637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5ef2c-902c-4ce1-a45f-7b52b0760c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:03.902285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5ef2c-902c-4ce1-a45f-7b52b0760c84 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:03.902569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5ef2c-902c-4ce1-a45f-7b52b0760c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:03.902846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5ef2c-902c-4ce1-a45f-7b52b0760c84 None admin] PATH_INFO: `/v3/projects/6b678825ee3d405fb7aea9381dc8d9fd/users/ed037216ad65437388bca1c9c4e28d3b/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:03.905019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5ef2c-902c-4ce1-a45f-7b52b0760c84 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b678825ee3d405fb7aea9381dc8d9fd, user_id=ed037216ad65437388bca1c9c4e28d3b, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:03.906460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5ef2c-902c-4ce1-a45f-7b52b0760c84 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:03.919748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 604/1192] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:12:03 2022] PUT /identity/v3/projects/6b678825ee3d405fb7aea9381dc8d9fd/users/ed037216ad65437388bca1c9c4e28d3b/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:12:05.788530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9fa44-2eb3-4145-bce0-d3787ddfd558 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:05.789075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9fa44-2eb3-4145-bce0-d3787ddfd558 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:05.789075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9fa44-2eb3-4145-bce0-d3787ddfd558 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:05.790891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 589/1193] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:12:05.796222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be411332-4112-44ac-96d7-a55511c6cc0b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:05.796694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be411332-4112-44ac-96d7-a55511c6cc0b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:05.796801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be411332-4112-44ac-96d7-a55511c6cc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:05.814970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-be411332-4112-44ac-96d7-a55511c6cc0b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:12:05.816145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be411332-4112-44ac-96d7-a55511c6cc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:12:05.829952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:05.882772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 605/1194] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:12:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:12:06.155339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9fa44-2eb3-4145-bce0-d3787ddfd558 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:06.254199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9fa44-2eb3-4145-bce0-d3787ddfd558 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:06.287450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ac600-844e-45e0-aeaf-19025080b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:06.288049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ac600-844e-45e0-aeaf-19025080b5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:06.288324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ac600-844e-45e0-aeaf-19025080b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:06.288618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ac600-844e-45e0-aeaf-19025080b5d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:06.300057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ac600-844e-45e0-aeaf-19025080b5d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-103ac600-844e-45e0-aeaf-19025080b5d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:12:06.304771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:12:06.305878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:12:06.306744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 590/1195] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:12:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:12:06.311959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be411332-4112-44ac-96d7-a55511c6cc0b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:06.412330 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be411332-4112-44ac-96d7-a55511c6cc0b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:06.446320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1b75a-f240-4218-b2ef-80419c05abb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:06.446963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b75a-f240-4218-b2ef-80419c05abb5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:06.447242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b75a-f240-4218-b2ef-80419c05abb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:06.447517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1b75a-f240-4218-b2ef-80419c05abb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:06.448176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1b75a-f240-4218-b2ef-80419c05abb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:12:06.448506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1b75a-f240-4218-b2ef-80419c05abb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:06.449884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1b75a-f240-4218-b2ef-80419c05abb5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:06.457669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 606/1196] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:12:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:06.463722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ac600-844e-45e0-aeaf-19025080b5d4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:06.522973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:06.564131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ac600-844e-45e0-aeaf-19025080b5d4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:06.598500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d7061-8c29-4ab9-a80e-afc3cb642a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:06.599131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d7061-8c29-4ab9-a80e-afc3cb642a2e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:06.599405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d7061-8c29-4ab9-a80e-afc3cb642a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:06.599672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d7061-8c29-4ab9-a80e-afc3cb642a2e None admin] PATH_INFO: `/v3/users/ed037216ad65437388bca1c9c4e28d3b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:06.600789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d7061-8c29-4ab9-a80e-afc3cb642a2e None admin] RBAC: Authorizing `identity:get_user(user_id=ed037216ad65437388bca1c9c4e28d3b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:06.602219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d7061-8c29-4ab9-a80e-afc3cb642a2e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:06.604276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 591/1197] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Aug 19 15:12:06 2022] GET /identity/v3/users/ed037216ad65437388bca1c9c4e28d3b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:12:06.610233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1b75a-f240-4218-b2ef-80419c05abb5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:06.625832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:06.713188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1b75a-f240-4218-b2ef-80419c05abb5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:06.745907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c75de-a6e3-40f4-a2d0-ae907c07d6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:06.746503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c75de-a6e3-40f4-a2d0-ae907c07d6cb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:06.746780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c75de-a6e3-40f4-a2d0-ae907c07d6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:06.747050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c75de-a6e3-40f4-a2d0-ae907c07d6cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:06.812284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c75de-a6e3-40f4-a2d0-ae907c07d6cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:06.813897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c75de-a6e3-40f4-a2d0-ae907c07d6cb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:06.827346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:06.911018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 607/1198] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:12:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:12:06.917628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d7061-8c29-4ab9-a80e-afc3cb642a2e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:07.016508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d7061-8c29-4ab9-a80e-afc3cb642a2e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:07.043010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:07.051155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d439611-b3b5-4ec4-bea9-4b2b28723caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:07.051773 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d439611-b3b5-4ec4-bea9-4b2b28723caf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:07.052053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d439611-b3b5-4ec4-bea9-4b2b28723caf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:07.052325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d439611-b3b5-4ec4-bea9-4b2b28723caf None admin] PATH_INFO: `/v3/projects/6b678825ee3d405fb7aea9381dc8d9fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:07.053458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d439611-b3b5-4ec4-bea9-4b2b28723caf None admin] RBAC: Authorizing `identity:get_project(project_id=6b678825ee3d405fb7aea9381dc8d9fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:07.054900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d439611-b3b5-4ec4-bea9-4b2b28723caf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:07.057301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 592/1199] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Aug 19 15:12:06 2022] GET /identity/v3/projects/6b678825ee3d405fb7aea9381dc8d9fd => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:07.063409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c75de-a6e3-40f4-a2d0-ae907c07d6cb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:12:07.160793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c75de-a6e3-40f4-a2d0-ae907c07d6cb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:12:07.174329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:07.195635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f5594-8638-44ab-b4df-e5305388def3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:12:07.196231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f5594-8638-44ab-b4df-e5305388def3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:12:07.196540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f5594-8638-44ab-b4df-e5305388def3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:12:07.196860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f5594-8638-44ab-b4df-e5305388def3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:12:07.197573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f5594-8638-44ab-b4df-e5305388def3 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=6b678825ee3d405fb7aea9381dc8d9fd, user.id=ed037216ad65437388bca1c9c4e28d3b) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:12:07.197882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f5594-8638-44ab-b4df-e5305388def3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:12:07.199315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f5594-8638-44ab-b4df-e5305388def3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:12:07.204978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 608/1200] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:12:07 2022] GET /identity/v3/role_assignments?user.id=ed037216ad65437388bca1c9c4e28d3b&scope.project.id=6b678825ee3d405fb7aea9381dc8d9fd&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:12:17.047784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:12:17.202372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=12 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:02.940935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400b23e-7cb5-4cb9-8731-f32a0a88a69c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:02.941481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400b23e-7cb5-4cb9-8731-f32a0a88a69c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:02.941481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400b23e-7cb5-4cb9-8731-f32a0a88a69c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:02.943165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 593/1201] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:02.949408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5236d-e113-489f-a00b-c142847dd33e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:02.949896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5236d-e113-489f-a00b-c142847dd33e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:02.949998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5236d-e113-489f-a00b-c142847dd33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:02.970828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d8f5236d-e113-489f-a00b-c142847dd33e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:02.972017 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8f5236d-e113-489f-a00b-c142847dd33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:03.061644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 609/1202] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:03.346336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400b23e-7cb5-4cb9-8731-f32a0a88a69c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:03.480334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400b23e-7cb5-4cb9-8731-f32a0a88a69c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:03.525938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c35c98-5e0a-491a-9a76-715dc99eab88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:03.526599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c35c98-5e0a-491a-9a76-715dc99eab88 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:03.526892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c35c98-5e0a-491a-9a76-715dc99eab88 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:03.527278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c35c98-5e0a-491a-9a76-715dc99eab88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:03.540684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c35c98-5e0a-491a-9a76-715dc99eab88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:03.542561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c35c98-5e0a-491a-9a76-715dc99eab88 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-79c35c98-5e0a-491a-9a76-715dc99eab88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:03.561050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:03.562177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:03.563840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:03.563840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:03.563840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:03.563840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 594/1203] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:13:03.568335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5236d-e113-489f-a00b-c142847dd33e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:03.710419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5236d-e113-489f-a00b-c142847dd33e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:03.753684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:03.763113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998851e-3df0-4236-8684-87033969b3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:03.764176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998851e-3df0-4236-8684-87033969b3ba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:03.764610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998851e-3df0-4236-8684-87033969b3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:03.764990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998851e-3df0-4236-8684-87033969b3ba None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:03.765793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998851e-3df0-4236-8684-87033969b3ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:03.766266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998851e-3df0-4236-8684-87033969b3ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:03.769968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998851e-3df0-4236-8684-87033969b3ba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:03.784653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 610/1204] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:03.790748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c35c98-5e0a-491a-9a76-715dc99eab88 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:03.862053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:03.929975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c35c98-5e0a-491a-9a76-715dc99eab88 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:03.967789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444111a4-2af7-473d-81b5-f3c548737e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:03.968659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444111a4-2af7-473d-81b5-f3c548737e58 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:03.968912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444111a4-2af7-473d-81b5-f3c548737e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:03.969181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444111a4-2af7-473d-81b5-f3c548737e58 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:03.969834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444111a4-2af7-473d-81b5-f3c548737e58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:03.971276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444111a4-2af7-473d-81b5-f3c548737e58 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:04.000327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 595/1205] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Aug 19 15:13:03 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:13:06.138492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37038471-7a33-4330-baf8-e8575f1f85f9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:06.139286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37038471-7a33-4330-baf8-e8575f1f85f9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:06.139286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37038471-7a33-4330-baf8-e8575f1f85f9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:06.140684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 611/1206] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:06.146193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bff4ac-074d-4bbf-888a-2b86062c95b4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:06.146637 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bff4ac-074d-4bbf-888a-2b86062c95b4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:06.146814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bff4ac-074d-4bbf-888a-2b86062c95b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:06.169419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f1bff4ac-074d-4bbf-888a-2b86062c95b4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:06.170618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f1bff4ac-074d-4bbf-888a-2b86062c95b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:06.252236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 596/1207] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:06.543896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37038471-7a33-4330-baf8-e8575f1f85f9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:06.673710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37038471-7a33-4330-baf8-e8575f1f85f9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:06.721771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e146-0ba9-4e77-9a76-09a38294452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:06.723039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e146-0ba9-4e77-9a76-09a38294452f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:06.723039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e146-0ba9-4e77-9a76-09a38294452f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:06.723217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0e146-0ba9-4e77-9a76-09a38294452f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:06.737183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0e146-0ba9-4e77-9a76-09a38294452f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-42d0e146-0ba9-4e77-9a76-09a38294452f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:13:06.741824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:06.742942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:06.745912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 612/1208] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:13:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:06.750864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bff4ac-074d-4bbf-888a-2b86062c95b4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:06.800014 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:06.878594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bff4ac-074d-4bbf-888a-2b86062c95b4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:06.922698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c38ac-9a10-46b0-a7b7-f712fcd26230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:06.923460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c38ac-9a10-46b0-a7b7-f712fcd26230 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:06.923745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c38ac-9a10-46b0-a7b7-f712fcd26230 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:06.924132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c38ac-9a10-46b0-a7b7-f712fcd26230 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:06.924849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c38ac-9a10-46b0-a7b7-f712fcd26230 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:06.925156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c38ac-9a10-46b0-a7b7-f712fcd26230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:06.926994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c38ac-9a10-46b0-a7b7-f712fcd26230 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:06.937158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 597/1209] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:13:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:06.943508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e146-0ba9-4e77-9a76-09a38294452f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:07.005509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:07.073845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0e146-0ba9-4e77-9a76-09a38294452f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:07.126530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeb0ee-43e6-4a13-98ef-a18737e3f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:07.127307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeb0ee-43e6-4a13-98ef-a18737e3f747 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:07.127586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeb0ee-43e6-4a13-98ef-a18737e3f747 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:07.127893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeb0ee-43e6-4a13-98ef-a18737e3f747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:07.143092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeb0ee-43e6-4a13-98ef-a18737e3f747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:07.144714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeb0ee-43e6-4a13-98ef-a18737e3f747 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d7aeb0ee-43e6-4a13-98ef-a18737e3f747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:07.162185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:07.163308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:07.169882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:07.169882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:07.169882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:07.169882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 613/1210] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:07.173282 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c38ac-9a10-46b0-a7b7-f712fcd26230 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:07.287708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:07.292351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c38ac-9a10-46b0-a7b7-f712fcd26230 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:07.333760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b7e18-1e16-4584-b8d2-128b12f87e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:07.334247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b7e18-1e16-4584-b8d2-128b12f87e62 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:07.334568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b7e18-1e16-4584-b8d2-128b12f87e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:07.334816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b7e18-1e16-4584-b8d2-128b12f87e62 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:07.335499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b7e18-1e16-4584-b8d2-128b12f87e62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:07.335871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b7e18-1e16-4584-b8d2-128b12f87e62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:07.338201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b7e18-1e16-4584-b8d2-128b12f87e62 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:07.352241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 598/1211] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:13:07.358579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeb0ee-43e6-4a13-98ef-a18737e3f747 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:07.468239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeb0ee-43e6-4a13-98ef-a18737e3f747 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:07.469971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:07.506437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4cfbe-f7d5-42d3-ac4b-5bc1c0805173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:07.507039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4cfbe-f7d5-42d3-ac4b-5bc1c0805173 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:07.507343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4cfbe-f7d5-42d3-ac4b-5bc1c0805173 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:07.507650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4cfbe-f7d5-42d3-ac4b-5bc1c0805173 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:07.516343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4cfbe-f7d5-42d3-ac4b-5bc1c0805173 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:07.517992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4cfbe-f7d5-42d3-ac4b-5bc1c0805173 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-02b4cfbe-f7d5-42d3-ac4b-5bc1c0805173 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:13:07.529866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:07.531229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:07.532918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 614/1212] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:13:07.537125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b7e18-1e16-4584-b8d2-128b12f87e62 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:07.648817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b7e18-1e16-4584-b8d2-128b12f87e62 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:07.688518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c72f3b-1fdc-46f7-8b4c-1c23e1c2ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:07.689137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c72f3b-1fdc-46f7-8b4c-1c23e1c2ff88 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:07.689477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c72f3b-1fdc-46f7-8b4c-1c23e1c2ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:07.689781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c72f3b-1fdc-46f7-8b4c-1c23e1c2ff88 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:07.691114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c72f3b-1fdc-46f7-8b4c-1c23e1c2ff88 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:07.691581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c72f3b-1fdc-46f7-8b4c-1c23e1c2ff88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:07.693080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c72f3b-1fdc-46f7-8b4c-1c23e1c2ff88 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:07.712877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 599/1213] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Aug 19 15:13:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:07.718604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4cfbe-f7d5-42d3-ac4b-5bc1c0805173 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:07.831168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4cfbe-f7d5-42d3-ac4b-5bc1c0805173 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:07.874714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71959f-7514-4bcb-b467-c3f0d989fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:07.875337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71959f-7514-4bcb-b467-c3f0d989fd02 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:07.875631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71959f-7514-4bcb-b467-c3f0d989fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:07.875908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71959f-7514-4bcb-b467-c3f0d989fd02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:07.910734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:07.956216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71959f-7514-4bcb-b467-c3f0d989fd02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:07.957804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71959f-7514-4bcb-b467-c3f0d989fd02 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:08.080522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 615/1214] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:13:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 15:13:08.086463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c72f3b-1fdc-46f7-8b4c-1c23e1c2ff88 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:08.113108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:08.217170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c72f3b-1fdc-46f7-8b4c-1c23e1c2ff88 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:08.266072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293fa91-8886-4a72-9a70-cd5233244531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:08.266948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293fa91-8886-4a72-9a70-cd5233244531 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:08.267333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293fa91-8886-4a72-9a70-cd5233244531 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:08.267713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293fa91-8886-4a72-9a70-cd5233244531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:08.286221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293fa91-8886-4a72-9a70-cd5233244531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:08.287932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293fa91-8886-4a72-9a70-cd5233244531 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-c293fa91-8886-4a72-9a70-cd5233244531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:08.305155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:08.307128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:08.308327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:08.308327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:08.308327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:08.312155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 600/1215] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:08.313367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71959f-7514-4bcb-b467-c3f0d989fd02 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:08.384052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:08.429474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71959f-7514-4bcb-b467-c3f0d989fd02 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:08.470648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba7495-7c2a-4953-b965-16f3908e29b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:08.471170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba7495-7c2a-4953-b965-16f3908e29b9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:08.471447 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba7495-7c2a-4953-b965-16f3908e29b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:08.471713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba7495-7c2a-4953-b965-16f3908e29b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:08.472354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba7495-7c2a-4953-b965-16f3908e29b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:08.472682 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba7495-7c2a-4953-b965-16f3908e29b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:08.474054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba7495-7c2a-4953-b965-16f3908e29b9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:08.492421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 616/1216] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:08.495010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293fa91-8886-4a72-9a70-cd5233244531 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:08.577756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:08.618962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293fa91-8886-4a72-9a70-cd5233244531 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:08.657963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5c1cd-ff8b-415f-9789-0cc64993e2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:08.658593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5c1cd-ff8b-415f-9789-0cc64993e2db None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:08.658867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5c1cd-ff8b-415f-9789-0cc64993e2db None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:08.659130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5c1cd-ff8b-415f-9789-0cc64993e2db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:08.666788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5c1cd-ff8b-415f-9789-0cc64993e2db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:08.668242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5c1cd-ff8b-415f-9789-0cc64993e2db None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-aac5c1cd-ff8b-415f-9789-0cc64993e2db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:13:08.685515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:08.686645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:08.687570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:13:08.687570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:13:08.687570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:13:08.687570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:13:08.687570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:13:08.687570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:13:08.687570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:08.687570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:08.687570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 601/1217] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:13:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:13:08.692518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba7495-7c2a-4953-b965-16f3908e29b9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:08.800552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba7495-7c2a-4953-b965-16f3908e29b9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:08.824146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:08.839766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2f409-0d3e-4f71-9183-a658e14066ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:08.840575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2f409-0d3e-4f71-9183-a658e14066ea None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:08.840949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2f409-0d3e-4f71-9183-a658e14066ea None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:08.841317 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2f409-0d3e-4f71-9183-a658e14066ea None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:08.842102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2f409-0d3e-4f71-9183-a658e14066ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:08.842546 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2f409-0d3e-4f71-9183-a658e14066ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:08.844251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2f409-0d3e-4f71-9183-a658e14066ea None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:08.859658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 617/1218] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:08.866362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5c1cd-ff8b-415f-9789-0cc64993e2db None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:08.987041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5c1cd-ff8b-415f-9789-0cc64993e2db None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:09.024487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:09.032616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c24b2-be1d-48d2-8e27-2e8308c4de44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:09.033449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c24b2-be1d-48d2-8e27-2e8308c4de44 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:09.033905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c24b2-be1d-48d2-8e27-2e8308c4de44 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:09.034315 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c24b2-be1d-48d2-8e27-2e8308c4de44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:09.035298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c24b2-be1d-48d2-8e27-2e8308c4de44 None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=bc2920aac4c046aca75473158ec204cc) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:09.035749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c24b2-be1d-48d2-8e27-2e8308c4de44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:09.037584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c24b2-be1d-48d2-8e27-2e8308c4de44 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:09.044551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 602/1219] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:13:08 2022] GET /identity/v3/role_assignments?user.id=bc2920aac4c046aca75473158ec204cc&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:11.033640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971896c5-9215-4178-ab64-f621b057a011 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:11.034203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971896c5-9215-4178-ab64-f621b057a011 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:11.034332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971896c5-9215-4178-ab64-f621b057a011 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:11.036291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 618/1220] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:11.043175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9a7ce-850f-4f88-8edd-06b7a34290d8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:11.043671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9a7ce-850f-4f88-8edd-06b7a34290d8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:11.043907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9a7ce-850f-4f88-8edd-06b7a34290d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:11.065609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b9c9a7ce-850f-4f88-8edd-06b7a34290d8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:11.066975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b9c9a7ce-850f-4f88-8edd-06b7a34290d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:11.145900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 603/1221] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:11.412229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971896c5-9215-4178-ab64-f621b057a011 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:11.525976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971896c5-9215-4178-ab64-f621b057a011 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:11.568073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc968f3-f682-4af1-9cb0-1141e519c907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:11.568738 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc968f3-f682-4af1-9cb0-1141e519c907 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:11.569021 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc968f3-f682-4af1-9cb0-1141e519c907 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:11.569295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc968f3-f682-4af1-9cb0-1141e519c907 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:11.579785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc968f3-f682-4af1-9cb0-1141e519c907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3bc968f3-f682-4af1-9cb0-1141e519c907 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:13:11.584684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:11.586056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:11.587258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 619/1222] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:13:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:11.592381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9a7ce-850f-4f88-8edd-06b7a34290d8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:11.702184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9a7ce-850f-4f88-8edd-06b7a34290d8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:11.727440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:11.738592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7abac6-4c69-408b-b020-feafb796b521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:11.739205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7abac6-4c69-408b-b020-feafb796b521 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:11.739487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7abac6-4c69-408b-b020-feafb796b521 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:11.739757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7abac6-4c69-408b-b020-feafb796b521 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:11.740456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7abac6-4c69-408b-b020-feafb796b521 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:11.740761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7abac6-4c69-408b-b020-feafb796b521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:11.742141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7abac6-4c69-408b-b020-feafb796b521 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:11.750689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 604/1223] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:13:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:11.756407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc968f3-f682-4af1-9cb0-1141e519c907 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:11.807485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:11.869421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc968f3-f682-4af1-9cb0-1141e519c907 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:11.906778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5f050-3f79-493b-8064-a3a3bf092e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:11.907399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5f050-3f79-493b-8064-a3a3bf092e10 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:11.907677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5f050-3f79-493b-8064-a3a3bf092e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:11.907949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5f050-3f79-493b-8064-a3a3bf092e10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:11.918585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5f050-3f79-493b-8064-a3a3bf092e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:11.920058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5f050-3f79-493b-8064-a3a3bf092e10 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bec5f050-3f79-493b-8064-a3a3bf092e10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:11.935843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:11.937855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:11.940129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:11.940129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:11.940129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:11.940129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 620/1224] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:11.944208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7abac6-4c69-408b-b020-feafb796b521 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:12.057884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7abac6-4c69-408b-b020-feafb796b521 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:12.096134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025674c-d522-43ed-9a0e-08d76760fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:12.097202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025674c-d522-43ed-9a0e-08d76760fe4c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:12.097661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025674c-d522-43ed-9a0e-08d76760fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:12.098190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025674c-d522-43ed-9a0e-08d76760fe4c None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:12.098954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025674c-d522-43ed-9a0e-08d76760fe4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:12.099377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025674c-d522-43ed-9a0e-08d76760fe4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:12.100885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025674c-d522-43ed-9a0e-08d76760fe4c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:12.114616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 605/1225] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:12.120759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5f050-3f79-493b-8064-a3a3bf092e10 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:12.227499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:12.236832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5f050-3f79-493b-8064-a3a3bf092e10 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:12.272715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8bfde-66b7-43b7-9797-6bfcf860edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:12.273349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8bfde-66b7-43b7-9797-6bfcf860edb3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:12.273627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8bfde-66b7-43b7-9797-6bfcf860edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:12.273899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8bfde-66b7-43b7-9797-6bfcf860edb3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:12.282619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8bfde-66b7-43b7-9797-6bfcf860edb3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:12.284127 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8bfde-66b7-43b7-9797-6bfcf860edb3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ccb8bfde-66b7-43b7-9797-6bfcf860edb3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:13:12.299475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:12.300710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:12.302024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 621/1226] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:12.307641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025674c-d522-43ed-9a0e-08d76760fe4c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:12.359750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:12.422683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025674c-d522-43ed-9a0e-08d76760fe4c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:12.464791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8a3b2-124d-4391-a3b3-f41003cdcaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:12.465396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8a3b2-124d-4391-a3b3-f41003cdcaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:12.465671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8a3b2-124d-4391-a3b3-f41003cdcaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:12.465940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8a3b2-124d-4391-a3b3-f41003cdcaf9 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:12.466665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8a3b2-124d-4391-a3b3-f41003cdcaf9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:12.466960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8a3b2-124d-4391-a3b3-f41003cdcaf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:12.468468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8a3b2-124d-4391-a3b3-f41003cdcaf9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:12.490878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 606/1227] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Aug 19 15:13:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:12.497928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8bfde-66b7-43b7-9797-6bfcf860edb3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:12.614078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8bfde-66b7-43b7-9797-6bfcf860edb3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:12.652306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b28e6e-5348-4f6d-a62f-41879cd22b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:12.653097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b28e6e-5348-4f6d-a62f-41879cd22b0a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:12.653413 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b28e6e-5348-4f6d-a62f-41879cd22b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:12.653727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b28e6e-5348-4f6d-a62f-41879cd22b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:12.666628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b28e6e-5348-4f6d-a62f-41879cd22b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:12.669638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b28e6e-5348-4f6d-a62f-41879cd22b0a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-43b28e6e-5348-4f6d-a62f-41879cd22b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:12.685013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:12.686081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:12.687060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:12.687060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:12.687060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:12.687060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 622/1228] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:12.693545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8a3b2-124d-4391-a3b3-f41003cdcaf9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:12.801914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8a3b2-124d-4391-a3b3-f41003cdcaf9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:12.804302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:12.841138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175b3d8-4632-428c-a524-8e6d46a0f9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:12.841733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175b3d8-4632-428c-a524-8e6d46a0f9fb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:12.842011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175b3d8-4632-428c-a524-8e6d46a0f9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:12.842276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175b3d8-4632-428c-a524-8e6d46a0f9fb None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:12.842912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175b3d8-4632-428c-a524-8e6d46a0f9fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:12.843208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175b3d8-4632-428c-a524-8e6d46a0f9fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:12.844606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175b3d8-4632-428c-a524-8e6d46a0f9fb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:12.858569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 607/1229] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:12.864205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b28e6e-5348-4f6d-a62f-41879cd22b0a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:12.873929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:12.983273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b28e6e-5348-4f6d-a62f-41879cd22b0a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:13.021598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e56e18-6ad9-401b-8a73-b2f4181532b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:13.022224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e56e18-6ad9-401b-8a73-b2f4181532b7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:13.022502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e56e18-6ad9-401b-8a73-b2f4181532b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:13.022789 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e56e18-6ad9-401b-8a73-b2f4181532b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:13.030227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e56e18-6ad9-401b-8a73-b2f4181532b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:13.031956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e56e18-6ad9-401b-8a73-b2f4181532b7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-41e56e18-6ad9-401b-8a73-b2f4181532b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:13:13.046837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:13.047964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:13.048935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:13:13.048935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:13:13.048935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:13:13.048935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:13:13.048935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:13:13.048935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:13:13.048935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:13.048935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:13.048935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 623/1230] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:13.053968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175b3d8-4632-428c-a524-8e6d46a0f9fb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:13.164593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175b3d8-4632-428c-a524-8e6d46a0f9fb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:13.201162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f5683-d7fd-4f8b-ad75-c732e1f6f331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:13.201763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f5683-d7fd-4f8b-ad75-c732e1f6f331 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:13.202038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f5683-d7fd-4f8b-ad75-c732e1f6f331 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:13.202325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f5683-d7fd-4f8b-ad75-c732e1f6f331 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:13.202992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f5683-d7fd-4f8b-ad75-c732e1f6f331 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:13.203291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f5683-d7fd-4f8b-ad75-c732e1f6f331 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:13.204760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f5683-d7fd-4f8b-ad75-c732e1f6f331 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:13.217015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 608/1231] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:13:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:13.223255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e56e18-6ad9-401b-8a73-b2f4181532b7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:13.298539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:13.342783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e56e18-6ad9-401b-8a73-b2f4181532b7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:13.379038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0893b4-3a06-4a78-9d48-d09687f2c614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:13.379698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0893b4-3a06-4a78-9d48-d09687f2c614 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:13.379976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0893b4-3a06-4a78-9d48-d09687f2c614 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:13.380348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0893b4-3a06-4a78-9d48-d09687f2c614 None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/bc2920aac4c046aca75473158ec204cc/roles/fc63a4da31e84ad78f04c587c0c1c83c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:13.396475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0893b4-3a06-4a78-9d48-d09687f2c614 None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=bc2920aac4c046aca75473158ec204cc, role_id=fc63a4da31e84ad78f04c587c0c1c83c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:13.397964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0893b4-3a06-4a78-9d48-d09687f2c614 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:13.416935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 624/1232] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:13:13 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/bc2920aac4c046aca75473158ec204cc/roles/fc63a4da31e84ad78f04c587c0c1c83c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:13:15.477117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf31e35-4736-48ed-b0cd-168835274285 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:15.477663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf31e35-4736-48ed-b0cd-168835274285 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:15.477663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf31e35-4736-48ed-b0cd-168835274285 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:15.479173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 609/1233] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:13:15.484431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41593f-060e-4cfb-890c-df16f9b0665e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:15.484853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41593f-060e-4cfb-890c-df16f9b0665e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:15.485011 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41593f-060e-4cfb-890c-df16f9b0665e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:15.503103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4b41593f-060e-4cfb-890c-df16f9b0665e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:15.504298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4b41593f-060e-4cfb-890c-df16f9b0665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:15.573292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 625/1234] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:15.881926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf31e35-4736-48ed-b0cd-168835274285 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:15.913290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:16.000667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf31e35-4736-48ed-b0cd-168835274285 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:16.047439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b790a-6f47-463f-ba63-1ce6ac74bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:16.048125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b790a-6f47-463f-ba63-1ce6ac74bab7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:16.048433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b790a-6f47-463f-ba63-1ce6ac74bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:16.048768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b790a-6f47-463f-ba63-1ce6ac74bab7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:16.061387 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b790a-6f47-463f-ba63-1ce6ac74bab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-738b790a-6f47-463f-ba63-1ce6ac74bab7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:13:16.066777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:16.067950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:16.069000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 610/1235] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Aug 19 15:13:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:16.075194 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41593f-060e-4cfb-890c-df16f9b0665e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:16.127804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:16.189179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41593f-060e-4cfb-890c-df16f9b0665e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:16.227763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a03e9-2f50-4914-ac5a-629bfa5434dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:16.228633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a03e9-2f50-4914-ac5a-629bfa5434dc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:16.228920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a03e9-2f50-4914-ac5a-629bfa5434dc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:16.229195 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a03e9-2f50-4914-ac5a-629bfa5434dc None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:16.229860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a03e9-2f50-4914-ac5a-629bfa5434dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:16.230161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a03e9-2f50-4914-ac5a-629bfa5434dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:16.231561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a03e9-2f50-4914-ac5a-629bfa5434dc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:16.240015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 626/1236] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Aug 19 15:13:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:16.246193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b790a-6f47-463f-ba63-1ce6ac74bab7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:16.338328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:16.355014 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b790a-6f47-463f-ba63-1ce6ac74bab7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:16.394629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02feaae7-cc25-4946-8691-50dd565ecb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:16.395287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02feaae7-cc25-4946-8691-50dd565ecb0e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:16.395568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02feaae7-cc25-4946-8691-50dd565ecb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:16.395876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02feaae7-cc25-4946-8691-50dd565ecb0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:16.408128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02feaae7-cc25-4946-8691-50dd565ecb0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:16.411542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02feaae7-cc25-4946-8691-50dd565ecb0e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-02feaae7-cc25-4946-8691-50dd565ecb0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:16.432462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:16.433694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:16.434646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:16.434646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:16.434646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:16.434646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 611/1237] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:16.439903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a03e9-2f50-4914-ac5a-629bfa5434dc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:16.554259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a03e9-2f50-4914-ac5a-629bfa5434dc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:16.565241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:16.595075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3b7a-31b0-41f3-8366-25720f024d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:16.595705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3b7a-31b0-41f3-8366-25720f024d2a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:16.595984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3b7a-31b0-41f3-8366-25720f024d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:16.596306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d3b7a-31b0-41f3-8366-25720f024d2a None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:16.597020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d3b7a-31b0-41f3-8366-25720f024d2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:16.597327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d3b7a-31b0-41f3-8366-25720f024d2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:16.598839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d3b7a-31b0-41f3-8366-25720f024d2a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:16.612909 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 627/1238] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:16.618998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02feaae7-cc25-4946-8691-50dd565ecb0e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:16.739075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02feaae7-cc25-4946-8691-50dd565ecb0e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:16.774981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bca58d-dd71-4699-98ab-6c7f3798689c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:16.775776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bca58d-dd71-4699-98ab-6c7f3798689c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:16.776059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bca58d-dd71-4699-98ab-6c7f3798689c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:16.776336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bca58d-dd71-4699-98ab-6c7f3798689c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:16.779069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:16.785149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bca58d-dd71-4699-98ab-6c7f3798689c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:16.786623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bca58d-dd71-4699-98ab-6c7f3798689c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-62bca58d-dd71-4699-98ab-6c7f3798689c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:13:16.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:16.800123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:16.801419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 612/1239] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:16.805777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3b7a-31b0-41f3-8366-25720f024d2a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:16.925109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d3b7a-31b0-41f3-8366-25720f024d2a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:16.966962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79830039-a15d-49c8-b316-c822a71dd174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:16.967638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79830039-a15d-49c8-b316-c822a71dd174 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:16.967917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79830039-a15d-49c8-b316-c822a71dd174 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:16.968193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79830039-a15d-49c8-b316-c822a71dd174 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:16.968952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79830039-a15d-49c8-b316-c822a71dd174 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:16.969259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79830039-a15d-49c8-b316-c822a71dd174 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:16.970721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79830039-a15d-49c8-b316-c822a71dd174 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:16.989678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 628/1240] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Aug 19 15:13:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:13:16.997125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bca58d-dd71-4699-98ab-6c7f3798689c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:17.129225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bca58d-dd71-4699-98ab-6c7f3798689c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:17.165698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6f431-8695-4725-81f3-efcea9a716d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:17.166461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6f431-8695-4725-81f3-efcea9a716d5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:17.166780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6f431-8695-4725-81f3-efcea9a716d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:17.167056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6f431-8695-4725-81f3-efcea9a716d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:17.237626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:17.245113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6f431-8695-4725-81f3-efcea9a716d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:17.246514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6f431-8695-4725-81f3-efcea9a716d5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:17.368458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 613/1241] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:13:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:13:17.371332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79830039-a15d-49c8-b316-c822a71dd174 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:17.404209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:17.483711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79830039-a15d-49c8-b316-c822a71dd174 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:17.520631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dc580-b5c2-4a75-9671-00356fe08581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:17.521261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dc580-b5c2-4a75-9671-00356fe08581 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:17.521534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dc580-b5c2-4a75-9671-00356fe08581 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:17.521802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dc580-b5c2-4a75-9671-00356fe08581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:17.533360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dc580-b5c2-4a75-9671-00356fe08581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:17.534803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dc580-b5c2-4a75-9671-00356fe08581 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8a9dc580-b5c2-4a75-9671-00356fe08581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:17.549626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:17.550655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:17.551551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:17.551551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:17.551551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:17.551551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 629/1242] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 15:13:17.557471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6f431-8695-4725-81f3-efcea9a716d5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:17.666405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:17.673472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6f431-8695-4725-81f3-efcea9a716d5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:17.714208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca291416-0446-404f-ad58-2bc7c601eb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:17.714813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca291416-0446-404f-ad58-2bc7c601eb28 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:17.715092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca291416-0446-404f-ad58-2bc7c601eb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:17.715363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca291416-0446-404f-ad58-2bc7c601eb28 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:17.716005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca291416-0446-404f-ad58-2bc7c601eb28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:17.716306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca291416-0446-404f-ad58-2bc7c601eb28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:17.717714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca291416-0446-404f-ad58-2bc7c601eb28 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:17.730082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 614/1243] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:17.736134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dc580-b5c2-4a75-9671-00356fe08581 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:17.826082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:17.849572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dc580-b5c2-4a75-9671-00356fe08581 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:17.887019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a213613-c9bd-44fb-9c77-db48a165e370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:17.887775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a213613-c9bd-44fb-9c77-db48a165e370 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:17.888124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a213613-c9bd-44fb-9c77-db48a165e370 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:17.888494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a213613-c9bd-44fb-9c77-db48a165e370 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:17.896163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a213613-c9bd-44fb-9c77-db48a165e370 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:17.897890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a213613-c9bd-44fb-9c77-db48a165e370 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7a213613-c9bd-44fb-9c77-db48a165e370 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:13:17.912730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:17.913856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:17.914949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:13:17.914949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:13:17.914949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:13:17.914949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:13:17.914949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:13:17.914949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:13:17.914949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:17.914949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:17.914949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 630/1244] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:17.920857 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca291416-0446-404f-ad58-2bc7c601eb28 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:18.033596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca291416-0446-404f-ad58-2bc7c601eb28 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:18.075685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0b78f-ad0a-4039-bf75-159f4f3fa46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:18.076351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0b78f-ad0a-4039-bf75-159f4f3fa46e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:18.076608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0b78f-ad0a-4039-bf75-159f4f3fa46e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:18.076880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0b78f-ad0a-4039-bf75-159f4f3fa46e None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:18.077575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0b78f-ad0a-4039-bf75-159f4f3fa46e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:18.077881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0b78f-ad0a-4039-bf75-159f4f3fa46e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:18.079324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0b78f-ad0a-4039-bf75-159f4f3fa46e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:18.093922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 615/1245] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:18.101151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a213613-c9bd-44fb-9c77-db48a165e370 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:18.225181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a213613-c9bd-44fb-9c77-db48a165e370 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:18.264808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6636d-9bf8-472d-b8d6-e695013ae6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:18.265420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6636d-9bf8-472d-b8d6-e695013ae6ef None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:18.265702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6636d-9bf8-472d-b8d6-e695013ae6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:18.265973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6636d-9bf8-472d-b8d6-e695013ae6ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:18.266701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6636d-9bf8-472d-b8d6-e695013ae6ef None admin] RBAC: Adding query filter params (role.id=fc63a4da31e84ad78f04c587c0c1c83c, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=bc2920aac4c046aca75473158ec204cc) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:18.267009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6636d-9bf8-472d-b8d6-e695013ae6ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:18.268478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6636d-9bf8-472d-b8d6-e695013ae6ef None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:18.276340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 631/1246] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:13:18 2022] GET /identity/v3/role_assignments?user.id=bc2920aac4c046aca75473158ec204cc&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=fc63a4da31e84ad78f04c587c0c1c83c => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:20.352034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57369ea-71ac-42cf-b275-5e3b5f4864c9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:20.352609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57369ea-71ac-42cf-b275-5e3b5f4864c9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:20.352609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57369ea-71ac-42cf-b275-5e3b5f4864c9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:20.354806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 616/1247] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:20.360276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc39b9a-756e-492b-849f-fdc0bce981f1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:20.360874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc39b9a-756e-492b-849f-fdc0bce981f1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:20.360874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc39b9a-756e-492b-849f-fdc0bce981f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:20.367082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:20.381846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-afc39b9a-756e-492b-849f-fdc0bce981f1 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:20.383030 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-afc39b9a-756e-492b-849f-fdc0bce981f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:20.463196 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 632/1248] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:20.764155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57369ea-71ac-42cf-b275-5e3b5f4864c9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:20.786430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:20.882639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57369ea-71ac-42cf-b275-5e3b5f4864c9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:20.927791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962d8a9-f25d-4540-9e99-0279344dac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:20.928450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962d8a9-f25d-4540-9e99-0279344dac4f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:20.928735 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962d8a9-f25d-4540-9e99-0279344dac4f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:20.929008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962d8a9-f25d-4540-9e99-0279344dac4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:20.942289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962d8a9-f25d-4540-9e99-0279344dac4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3962d8a9-f25d-4540-9e99-0279344dac4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:13:20.947761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:20.948921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:20.949821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 617/1249] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:13:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:20.955336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc39b9a-756e-492b-849f-fdc0bce981f1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:21.083104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc39b9a-756e-492b-849f-fdc0bce981f1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:21.120145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:21.123415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c829a6c-1cce-4fd3-889d-ff4736b4adab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:21.124122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c829a6c-1cce-4fd3-889d-ff4736b4adab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:21.124652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c829a6c-1cce-4fd3-889d-ff4736b4adab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:21.125068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c829a6c-1cce-4fd3-889d-ff4736b4adab None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:21.126247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c829a6c-1cce-4fd3-889d-ff4736b4adab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:21.126247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c829a6c-1cce-4fd3-889d-ff4736b4adab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:21.127845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c829a6c-1cce-4fd3-889d-ff4736b4adab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:21.136998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 633/1250] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:13:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:21.142813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962d8a9-f25d-4540-9e99-0279344dac4f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:21.236887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:21.264848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962d8a9-f25d-4540-9e99-0279344dac4f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:21.308853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539f2bd-ecfe-48ec-ad02-46804168b956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:21.309484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539f2bd-ecfe-48ec-ad02-46804168b956 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:21.309757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539f2bd-ecfe-48ec-ad02-46804168b956 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:21.310982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539f2bd-ecfe-48ec-ad02-46804168b956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:21.320738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539f2bd-ecfe-48ec-ad02-46804168b956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:21.322605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539f2bd-ecfe-48ec-ad02-46804168b956 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f539f2bd-ecfe-48ec-ad02-46804168b956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:21.337820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:21.339003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:21.340471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:21.340471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:21.340471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:21.340471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 618/1251] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:21.345204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c829a6c-1cce-4fd3-889d-ff4736b4adab None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:21.462212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c829a6c-1cce-4fd3-889d-ff4736b4adab None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:21.501957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42931b-c849-4de1-96a2-fa0e339d4f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:21.502465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42931b-c849-4de1-96a2-fa0e339d4f38 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:21.502744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42931b-c849-4de1-96a2-fa0e339d4f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:21.503004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42931b-c849-4de1-96a2-fa0e339d4f38 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:21.503645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42931b-c849-4de1-96a2-fa0e339d4f38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:21.503943 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42931b-c849-4de1-96a2-fa0e339d4f38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:21.505346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42931b-c849-4de1-96a2-fa0e339d4f38 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:21.518444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 634/1252] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:21.524622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539f2bd-ecfe-48ec-ad02-46804168b956 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:21.631864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539f2bd-ecfe-48ec-ad02-46804168b956 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:21.662930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:21.673581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cc08a-a332-4ff5-bfe7-5aee77cb957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:21.674463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cc08a-a332-4ff5-bfe7-5aee77cb957b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:21.675456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cc08a-a332-4ff5-bfe7-5aee77cb957b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:21.675873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cc08a-a332-4ff5-bfe7-5aee77cb957b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:21.684926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cc08a-a332-4ff5-bfe7-5aee77cb957b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:21.686377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24cc08a-a332-4ff5-bfe7-5aee77cb957b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e24cc08a-a332-4ff5-bfe7-5aee77cb957b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:13:21.698442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:21.705067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:21.706066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 619/1253] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:21.706676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42931b-c849-4de1-96a2-fa0e339d4f38 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:21.771949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:21.824653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42931b-c849-4de1-96a2-fa0e339d4f38 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:21.863808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5e7bd-77cd-4181-9f56-12cf3170b152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:21.864485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5e7bd-77cd-4181-9f56-12cf3170b152 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:21.864799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5e7bd-77cd-4181-9f56-12cf3170b152 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:21.864985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5e7bd-77cd-4181-9f56-12cf3170b152 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:21.865709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5e7bd-77cd-4181-9f56-12cf3170b152 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:21.866015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5e7bd-77cd-4181-9f56-12cf3170b152 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:21.867465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e5e7bd-77cd-4181-9f56-12cf3170b152 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:21.884246 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 635/1254] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Aug 19 15:13:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:21.890113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cc08a-a332-4ff5-bfe7-5aee77cb957b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:22.002043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cc08a-a332-4ff5-bfe7-5aee77cb957b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:22.045128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44274af-a8bf-4827-b731-61571460949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:22.045778 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44274af-a8bf-4827-b731-61571460949f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:22.046189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44274af-a8bf-4827-b731-61571460949f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:22.046553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44274af-a8bf-4827-b731-61571460949f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:22.106706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:22.120310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44274af-a8bf-4827-b731-61571460949f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:22.121802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44274af-a8bf-4827-b731-61571460949f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:22.230544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 620/1255] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:13:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:13:22.236486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5e7bd-77cd-4181-9f56-12cf3170b152 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:22.349735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5e7bd-77cd-4181-9f56-12cf3170b152 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:22.363955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:22.387343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e3311-9a4e-4109-8b1e-35d5b189d940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:22.388005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e3311-9a4e-4109-8b1e-35d5b189d940 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:22.388284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e3311-9a4e-4109-8b1e-35d5b189d940 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:22.388597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e3311-9a4e-4109-8b1e-35d5b189d940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:22.398279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e3311-9a4e-4109-8b1e-35d5b189d940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:22.399720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e3311-9a4e-4109-8b1e-35d5b189d940 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-609e3311-9a4e-4109-8b1e-35d5b189d940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:22.414502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:22.415841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:22.416955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:22.416955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:22.416955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:22.416955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 636/1256] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:22.421956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44274af-a8bf-4827-b731-61571460949f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:22.519465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:22.533545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44274af-a8bf-4827-b731-61571460949f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:22.572646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdd3e3-c71f-4447-8c29-c272e3658377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:22.573253 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdd3e3-c71f-4447-8c29-c272e3658377 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:22.573534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdd3e3-c71f-4447-8c29-c272e3658377 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:22.573808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdd3e3-c71f-4447-8c29-c272e3658377 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:22.574465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdd3e3-c71f-4447-8c29-c272e3658377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:22.574769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdd3e3-c71f-4447-8c29-c272e3658377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:22.576151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdd3e3-c71f-4447-8c29-c272e3658377 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:22.588103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 621/1257] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:22.593475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e3311-9a4e-4109-8b1e-35d5b189d940 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:22.702041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e3311-9a4e-4109-8b1e-35d5b189d940 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:22.737695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d70d3-dc6a-4e97-8f57-8ac6ed43461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:22.738326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d70d3-dc6a-4e97-8f57-8ac6ed43461c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:22.738604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d70d3-dc6a-4e97-8f57-8ac6ed43461c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:22.738869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d70d3-dc6a-4e97-8f57-8ac6ed43461c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:22.746188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d70d3-dc6a-4e97-8f57-8ac6ed43461c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:22.747671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d70d3-dc6a-4e97-8f57-8ac6ed43461c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4a7d70d3-dc6a-4e97-8f57-8ac6ed43461c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:13:22.761025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:22.762378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:22.763424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:13:22.763424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:13:22.763424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:13:22.763424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:13:22.763424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:13:22.763424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:13:22.763424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:22.763424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:22.767650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 637/1258] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:22.768682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdd3e3-c71f-4447-8c29-c272e3658377 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:22.878983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdd3e3-c71f-4447-8c29-c272e3658377 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:22.915304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930ece3-7f73-41e7-a6fd-1b875f8532a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:22.916690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930ece3-7f73-41e7-a6fd-1b875f8532a8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:22.917053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930ece3-7f73-41e7-a6fd-1b875f8532a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:22.917297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930ece3-7f73-41e7-a6fd-1b875f8532a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:22.918202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930ece3-7f73-41e7-a6fd-1b875f8532a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:22.918512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930ece3-7f73-41e7-a6fd-1b875f8532a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:22.920268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930ece3-7f73-41e7-a6fd-1b875f8532a8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:22.937521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 622/1259] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:13:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:13:22.944280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d70d3-dc6a-4e97-8f57-8ac6ed43461c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:22.974623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:23.069605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d70d3-dc6a-4e97-8f57-8ac6ed43461c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:23.110827 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2920a6a-9b42-4f04-bd3c-af506a25a2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:23.111638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2920a6a-9b42-4f04-bd3c-af506a25a2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:23.111919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2920a6a-9b42-4f04-bd3c-af506a25a2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:23.112242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2920a6a-9b42-4f04-bd3c-af506a25a2f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:23.112972 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2920a6a-9b42-4f04-bd3c-af506a25a2f9 None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=bc2920aac4c046aca75473158ec204cc) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:23.113266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2920a6a-9b42-4f04-bd3c-af506a25a2f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:23.114700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2920a6a-9b42-4f04-bd3c-af506a25a2f9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:23.121417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 638/1260] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:13:22 2022] GET /identity/v3/role_assignments?user.id=bc2920aac4c046aca75473158ec204cc&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:13:25.033080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8e4e1-efd8-411a-960b-fa670af74cab None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:25.033645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8e4e1-efd8-411a-960b-fa670af74cab None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:25.033645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8e4e1-efd8-411a-960b-fa670af74cab None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:25.035487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 623/1261] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:25.041140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8493e7e-4a03-4b7d-80b5-afb416816fa8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:25.041140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8493e7e-4a03-4b7d-80b5-afb416816fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:25.041778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8493e7e-4a03-4b7d-80b5-afb416816fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:25.059081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c8493e7e-4a03-4b7d-80b5-afb416816fa8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:25.060258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c8493e7e-4a03-4b7d-80b5-afb416816fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:25.125171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 639/1262] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:25.394451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8e4e1-efd8-411a-960b-fa670af74cab None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:25.405395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:25.493348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8e4e1-efd8-411a-960b-fa670af74cab None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:25.526879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b167e0d-4396-4006-bb33-4b8ed032caee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:25.527510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b167e0d-4396-4006-bb33-4b8ed032caee None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:25.527792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b167e0d-4396-4006-bb33-4b8ed032caee None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:25.528090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b167e0d-4396-4006-bb33-4b8ed032caee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:25.538295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b167e0d-4396-4006-bb33-4b8ed032caee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8b167e0d-4396-4006-bb33-4b8ed032caee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:13:25.543071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:25.544239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:25.545345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 624/1263] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:13:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:25.550117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8493e7e-4a03-4b7d-80b5-afb416816fa8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:25.568268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:25.647833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8493e7e-4a03-4b7d-80b5-afb416816fa8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:25.682862 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7606bf8-4814-4578-97a6-9b03a01620d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:25.683472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7606bf8-4814-4578-97a6-9b03a01620d9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:25.683753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7606bf8-4814-4578-97a6-9b03a01620d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:25.684027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7606bf8-4814-4578-97a6-9b03a01620d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:25.684730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7606bf8-4814-4578-97a6-9b03a01620d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:25.685037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7606bf8-4814-4578-97a6-9b03a01620d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:25.686425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7606bf8-4814-4578-97a6-9b03a01620d9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:25.694490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 640/1264] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:13:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:25.700891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b167e0d-4396-4006-bb33-4b8ed032caee None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:25.765748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:25.798094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b167e0d-4396-4006-bb33-4b8ed032caee None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:25.831403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6969d-c35e-41dc-8bfa-9f952d19faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:25.832012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6969d-c35e-41dc-8bfa-9f952d19faba None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:25.832286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6969d-c35e-41dc-8bfa-9f952d19faba None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:25.832580 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6969d-c35e-41dc-8bfa-9f952d19faba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:25.842922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6969d-c35e-41dc-8bfa-9f952d19faba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:25.844366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6969d-c35e-41dc-8bfa-9f952d19faba None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9ed6969d-c35e-41dc-8bfa-9f952d19faba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:25.858619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:25.859785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:25.860744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:25.860744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:25.860744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:25.860744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 625/1265] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:25.865527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7606bf8-4814-4578-97a6-9b03a01620d9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:25.949043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:25.967674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7606bf8-4814-4578-97a6-9b03a01620d9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:26.003761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0c634-fb88-43fa-9647-5ec43a7093d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:26.004365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0c634-fb88-43fa-9647-5ec43a7093d8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:26.004668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0c634-fb88-43fa-9647-5ec43a7093d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:26.004978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0c634-fb88-43fa-9647-5ec43a7093d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:26.005610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0c634-fb88-43fa-9647-5ec43a7093d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:26.005912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0c634-fb88-43fa-9647-5ec43a7093d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:26.007278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0c634-fb88-43fa-9647-5ec43a7093d8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:26.021017 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 641/1266] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:26.027335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6969d-c35e-41dc-8bfa-9f952d19faba None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:26.131460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6969d-c35e-41dc-8bfa-9f952d19faba None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:26.151272 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:26.166220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145afbb-a051-488e-a53b-29a3829f0ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:26.166813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145afbb-a051-488e-a53b-29a3829f0ecd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:26.167095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145afbb-a051-488e-a53b-29a3829f0ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:26.167369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145afbb-a051-488e-a53b-29a3829f0ecd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:26.174669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145afbb-a051-488e-a53b-29a3829f0ecd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:26.176116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145afbb-a051-488e-a53b-29a3829f0ecd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-c145afbb-a051-488e-a53b-29a3829f0ecd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:13:26.187860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:26.189109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:26.190290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 626/1267] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:26.197878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0c634-fb88-43fa-9647-5ec43a7093d8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:26.294192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0c634-fb88-43fa-9647-5ec43a7093d8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:26.327507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661eb0db-750d-46cd-8157-e020fcf8593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:26.328113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb0db-750d-46cd-8157-e020fcf8593b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:26.328374 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb0db-750d-46cd-8157-e020fcf8593b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:26.328678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb0db-750d-46cd-8157-e020fcf8593b None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:26.329391 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661eb0db-750d-46cd-8157-e020fcf8593b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:26.329692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661eb0db-750d-46cd-8157-e020fcf8593b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:26.331125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661eb0db-750d-46cd-8157-e020fcf8593b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:26.339715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:26.348055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 642/1268] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Aug 19 15:13:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:26.354093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145afbb-a051-488e-a53b-29a3829f0ecd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:26.452207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145afbb-a051-488e-a53b-29a3829f0ecd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:26.484941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f503c31-bc73-488a-af4c-11296e3fcead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:26.485546 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f503c31-bc73-488a-af4c-11296e3fcead None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:26.485822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f503c31-bc73-488a-af4c-11296e3fcead None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:26.486091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f503c31-bc73-488a-af4c-11296e3fcead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:26.497096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f503c31-bc73-488a-af4c-11296e3fcead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:26.498538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f503c31-bc73-488a-af4c-11296e3fcead None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3f503c31-bc73-488a-af4c-11296e3fcead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:26.512248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:26.513384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:26.514496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:26.514496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:26.514496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:26.514496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 627/1269] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:26.519253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661eb0db-750d-46cd-8157-e020fcf8593b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:26.616819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661eb0db-750d-46cd-8157-e020fcf8593b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:26.649535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9075f3-13c5-4fcb-af16-ec65483686fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:26.650231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9075f3-13c5-4fcb-af16-ec65483686fe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:26.650604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9075f3-13c5-4fcb-af16-ec65483686fe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:26.650875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9075f3-13c5-4fcb-af16-ec65483686fe None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:26.651508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9075f3-13c5-4fcb-af16-ec65483686fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:26.651805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9075f3-13c5-4fcb-af16-ec65483686fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:26.653198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9075f3-13c5-4fcb-af16-ec65483686fe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:26.664222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 643/1270] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:26.670633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f503c31-bc73-488a-af4c-11296e3fcead None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:26.702131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:26.771030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f503c31-bc73-488a-af4c-11296e3fcead None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:26.804064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052f0ed-ad81-4754-a1a0-4cf289d4db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:26.804717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052f0ed-ad81-4754-a1a0-4cf289d4db91 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:26.804999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052f0ed-ad81-4754-a1a0-4cf289d4db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:26.805274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052f0ed-ad81-4754-a1a0-4cf289d4db91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:26.811852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052f0ed-ad81-4754-a1a0-4cf289d4db91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:26.813351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052f0ed-ad81-4754-a1a0-4cf289d4db91 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-7052f0ed-ad81-4754-a1a0-4cf289d4db91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:13:26.824811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:26.825826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:26.826878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:13:26.826878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:13:26.826878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:13:26.826878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:13:26.826878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:13:26.826878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:13:26.826878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:26.826878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:26.826878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 628/1271] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:26.832343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9075f3-13c5-4fcb-af16-ec65483686fe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:26.890987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:26.934710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9075f3-13c5-4fcb-af16-ec65483686fe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:26.967502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576840d9-87a5-40ef-b7c6-2a8fe4c5dd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:26.968098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576840d9-87a5-40ef-b7c6-2a8fe4c5dd9a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:26.968379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576840d9-87a5-40ef-b7c6-2a8fe4c5dd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:26.968680 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576840d9-87a5-40ef-b7c6-2a8fe4c5dd9a None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:26.969364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576840d9-87a5-40ef-b7c6-2a8fe4c5dd9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:26.969668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576840d9-87a5-40ef-b7c6-2a8fe4c5dd9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:26.971102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576840d9-87a5-40ef-b7c6-2a8fe4c5dd9a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:26.982709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 644/1272] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:26.988820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052f0ed-ad81-4754-a1a0-4cf289d4db91 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:27.083436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:27.092530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052f0ed-ad81-4754-a1a0-4cf289d4db91 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:27.125437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1e2c1-68bd-4974-8c03-c788678a75d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:27.126155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1e2c1-68bd-4974-8c03-c788678a75d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:27.126445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1e2c1-68bd-4974-8c03-c788678a75d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:27.126754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1e2c1-68bd-4974-8c03-c788678a75d6 None admin] PATH_INFO: `/v3/projects/2520016785f144f5be02dae999d3037a/users/bc2920aac4c046aca75473158ec204cc/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:27.128775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1e2c1-68bd-4974-8c03-c788678a75d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2520016785f144f5be02dae999d3037a, user_id=bc2920aac4c046aca75473158ec204cc, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:27.130218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1e2c1-68bd-4974-8c03-c788678a75d6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:27.143034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 629/1273] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Aug 19 15:13:26 2022] PUT /identity/v3/projects/2520016785f144f5be02dae999d3037a/users/bc2920aac4c046aca75473158ec204cc/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:13:29.015288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9119e44-7c7c-402c-bf6b-340b60640a89 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:29.015942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9119e44-7c7c-402c-bf6b-340b60640a89 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:29.015942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9119e44-7c7c-402c-bf6b-340b60640a89 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:29.017370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 645/1274] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:29.022986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32b508-809a-4ae2-8515-5d4abedd0815 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:29.023417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32b508-809a-4ae2-8515-5d4abedd0815 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:29.023521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32b508-809a-4ae2-8515-5d4abedd0815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:29.044946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8c32b508-809a-4ae2-8515-5d4abedd0815 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:29.046281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8c32b508-809a-4ae2-8515-5d4abedd0815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:29.111230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 630/1275] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:29.390513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9119e44-7c7c-402c-bf6b-340b60640a89 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:29.489750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9119e44-7c7c-402c-bf6b-340b60640a89 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:29.506513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:29.523511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2ba08-869e-4c67-95cd-5eec6ec9c8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:29.524128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2ba08-869e-4c67-95cd-5eec6ec9c8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:29.524425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2ba08-869e-4c67-95cd-5eec6ec9c8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:29.524700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2ba08-869e-4c67-95cd-5eec6ec9c8c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:29.535604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2ba08-869e-4c67-95cd-5eec6ec9c8c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8ac2ba08-869e-4c67-95cd-5eec6ec9c8c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 15:13:29.540230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:29.541580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:29.542822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 646/1276] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Aug 19 15:13:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:29.547473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32b508-809a-4ae2-8515-5d4abedd0815 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:29.626333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:29.645941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32b508-809a-4ae2-8515-5d4abedd0815 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:29.681979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ebd65-1bfe-414a-b566-c8a492845011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:29.682591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ebd65-1bfe-414a-b566-c8a492845011 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:29.682865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ebd65-1bfe-414a-b566-c8a492845011 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:29.683131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ebd65-1bfe-414a-b566-c8a492845011 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:29.683791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ebd65-1bfe-414a-b566-c8a492845011 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:29.684089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ebd65-1bfe-414a-b566-c8a492845011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:29.685506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ebd65-1bfe-414a-b566-c8a492845011 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:29.693473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 631/1277] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Aug 19 15:13:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:29.699111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2ba08-869e-4c67-95cd-5eec6ec9c8c0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:29.799165 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2ba08-869e-4c67-95cd-5eec6ec9c8c0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:29.835065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ad3ab-41eb-411a-92f5-3462425d02ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:29.835688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ad3ab-41eb-411a-92f5-3462425d02ba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:29.835967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ad3ab-41eb-411a-92f5-3462425d02ba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:29.836238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ad3ab-41eb-411a-92f5-3462425d02ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:29.846715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ad3ab-41eb-411a-92f5-3462425d02ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:29.848182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ad3ab-41eb-411a-92f5-3462425d02ba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-473ad3ab-41eb-411a-92f5-3462425d02ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:29.862444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:29.863587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:29.864767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:29.864767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:29.864767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:29.864767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 647/1278] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:29.870595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ebd65-1bfe-414a-b566-c8a492845011 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:29.971264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ebd65-1bfe-414a-b566-c8a492845011 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:30.005040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31009db4-eb77-458e-be6d-066e906e814f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:30.005590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31009db4-eb77-458e-be6d-066e906e814f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:30.005863 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31009db4-eb77-458e-be6d-066e906e814f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:30.006229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31009db4-eb77-458e-be6d-066e906e814f None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:30.006815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31009db4-eb77-458e-be6d-066e906e814f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:30.007107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31009db4-eb77-458e-be6d-066e906e814f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:30.008506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31009db4-eb77-458e-be6d-066e906e814f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:30.020917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 632/1279] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:30.027256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ad3ab-41eb-411a-92f5-3462425d02ba None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:30.058339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:30.126510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ad3ab-41eb-411a-92f5-3462425d02ba None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:30.159120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da17d7-7555-4a58-8f6a-8cd6469a2ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:30.159896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da17d7-7555-4a58-8f6a-8cd6469a2ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:30.160178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da17d7-7555-4a58-8f6a-8cd6469a2ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:30.160475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da17d7-7555-4a58-8f6a-8cd6469a2ea3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:30.169320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da17d7-7555-4a58-8f6a-8cd6469a2ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:30.171012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da17d7-7555-4a58-8f6a-8cd6469a2ea3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-54da17d7-7555-4a58-8f6a-8cd6469a2ea3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 15:13:30.181899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:30.183305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:30.184498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 648/1280] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:30.188929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31009db4-eb77-458e-be6d-066e906e814f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:30.192812 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:30.290793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31009db4-eb77-458e-be6d-066e906e814f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:30.328053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448a086-c55f-4f83-b36e-2f78c311db13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:30.328689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a086-c55f-4f83-b36e-2f78c311db13 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:30.328973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a086-c55f-4f83-b36e-2f78c311db13 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:30.329247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a086-c55f-4f83-b36e-2f78c311db13 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:30.329987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448a086-c55f-4f83-b36e-2f78c311db13 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:30.330291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448a086-c55f-4f83-b36e-2f78c311db13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:30.331736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448a086-c55f-4f83-b36e-2f78c311db13 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:30.351752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 633/1281] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Aug 19 15:13:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:30.357805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da17d7-7555-4a58-8f6a-8cd6469a2ea3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:30.456574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:30.467276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da17d7-7555-4a58-8f6a-8cd6469a2ea3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:30.506101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565bbb-b13a-477f-ad32-5548fd979dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:30.506713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565bbb-b13a-477f-ad32-5548fd979dce None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:30.506988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565bbb-b13a-477f-ad32-5548fd979dce None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:30.507256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565bbb-b13a-477f-ad32-5548fd979dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:30.601584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4565bbb-b13a-477f-ad32-5548fd979dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:30.602980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4565bbb-b13a-477f-ad32-5548fd979dce None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:30.720823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:30.731989 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 649/1282] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Aug 19 15:13:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 15:13:30.737756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448a086-c55f-4f83-b36e-2f78c311db13 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:30.801257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:30.842784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448a086-c55f-4f83-b36e-2f78c311db13 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:30.877154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232e6aa-c69e-444c-9044-75731be24942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:30.877689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232e6aa-c69e-444c-9044-75731be24942 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:30.877967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232e6aa-c69e-444c-9044-75731be24942 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:30.878238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232e6aa-c69e-444c-9044-75731be24942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:30.889252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232e6aa-c69e-444c-9044-75731be24942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:30.890712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232e6aa-c69e-444c-9044-75731be24942 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b232e6aa-c69e-444c-9044-75731be24942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 15:13:30.905704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:30.911750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 15:13:30.912786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 15:13:30.912786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 15:13:30.912786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:30.912786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 634/1283] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:30.913048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565bbb-b13a-477f-ad32-5548fd979dce None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:31.028213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565bbb-b13a-477f-ad32-5548fd979dce None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:31.067753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae968-fbe3-4c21-8d97-bba24c8b5ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:31.068366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae968-fbe3-4c21-8d97-bba24c8b5ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:31.068678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae968-fbe3-4c21-8d97-bba24c8b5ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:31.068950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae968-fbe3-4c21-8d97-bba24c8b5ff7 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:31.069593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae968-fbe3-4c21-8d97-bba24c8b5ff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:31.069896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae968-fbe3-4c21-8d97-bba24c8b5ff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:31.071281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae968-fbe3-4c21-8d97-bba24c8b5ff7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:31.082898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 650/1284] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Aug 19 15:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:31.089344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232e6aa-c69e-444c-9044-75731be24942 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:31.187498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232e6aa-c69e-444c-9044-75731be24942 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:31.201043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:31.221278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f83ea-5f67-40ea-bb52-45f5581ff7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:31.221894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f83ea-5f67-40ea-bb52-45f5581ff7da None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:31.222172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f83ea-5f67-40ea-bb52-45f5581ff7da None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:31.222443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f83ea-5f67-40ea-bb52-45f5581ff7da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:31.228946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f83ea-5f67-40ea-bb52-45f5581ff7da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:31.230359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f83ea-5f67-40ea-bb52-45f5581ff7da None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-1e1f83ea-5f67-40ea-bb52-45f5581ff7da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 15:13:31.241183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:31.242227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:31.243577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 15:13:31.243577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 15:13:31.243577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 15:13:31.243577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 15:13:31.243577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 15:13:31.243577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 15:13:31.243577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 15:13:31.243577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Aug 19 15:13:31.243577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 635/1285] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Aug 19 15:13:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:31.248537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae968-fbe3-4c21-8d97-bba24c8b5ff7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:31.299427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:31.350357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae968-fbe3-4c21-8d97-bba24c8b5ff7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:31.383131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfae33c-893c-4dfb-9edc-2759de702b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:31.383731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfae33c-893c-4dfb-9edc-2759de702b12 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:31.384013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfae33c-893c-4dfb-9edc-2759de702b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:31.384285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfae33c-893c-4dfb-9edc-2759de702b12 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:31.385026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfae33c-893c-4dfb-9edc-2759de702b12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:31.385511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfae33c-893c-4dfb-9edc-2759de702b12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:31.387384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfae33c-893c-4dfb-9edc-2759de702b12 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:31.399323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 651/1286] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Aug 19 15:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:31.405933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f83ea-5f67-40ea-bb52-45f5581ff7da None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:31.503137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f83ea-5f67-40ea-bb52-45f5581ff7da None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:31.535591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9bab8-4723-4314-bc37-3eac239e7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:31.536181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9bab8-4723-4314-bc37-3eac239e7bdf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:31.536489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9bab8-4723-4314-bc37-3eac239e7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:31.536759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9bab8-4723-4314-bc37-3eac239e7bdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:31.537479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9bab8-4723-4314-bc37-3eac239e7bdf None admin] RBAC: Adding query filter params (role.id=52514e32ecde4219ab61ab4cbe90a063, scope.project.id=2520016785f144f5be02dae999d3037a, user.id=bc2920aac4c046aca75473158ec204cc) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:31.537789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9bab8-4723-4314-bc37-3eac239e7bdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:31.539219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9bab8-4723-4314-bc37-3eac239e7bdf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:31.545781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 636/1287] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Aug 19 15:13:31 2022] GET /identity/v3/role_assignments?user.id=bc2920aac4c046aca75473158ec204cc&scope.project.id=2520016785f144f5be02dae999d3037a&role.id=52514e32ecde4219ab61ab4cbe90a063 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:33.454796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c1882-e632-4236-a3d1-dbe0c1bf9cf5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:33.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c1882-e632-4236-a3d1-dbe0c1bf9cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:33.455327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c1882-e632-4236-a3d1-dbe0c1bf9cf5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:33.456677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 652/1288] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:33.461682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afee0e-2bc6-48c0-bb00-a01c80a2989b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:33.462080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afee0e-2bc6-48c0-bb00-a01c80a2989b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:33.462238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afee0e-2bc6-48c0-bb00-a01c80a2989b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:33.480453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-34afee0e-2bc6-48c0-bb00-a01c80a2989b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:33.481633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-34afee0e-2bc6-48c0-bb00-a01c80a2989b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:33.510989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:33.548070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 637/1289] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 15:13:33.812987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c1882-e632-4236-a3d1-dbe0c1bf9cf5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:33.913919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c1882-e632-4236-a3d1-dbe0c1bf9cf5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:33.925201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:33.949055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584ea3f-8dc3-4830-a8bb-405efaf31109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:33.949681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584ea3f-8dc3-4830-a8bb-405efaf31109 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:33.949957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584ea3f-8dc3-4830-a8bb-405efaf31109 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:33.950232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584ea3f-8dc3-4830-a8bb-405efaf31109 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:33.950802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584ea3f-8dc3-4830-a8bb-405efaf31109 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:33.952170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584ea3f-8dc3-4830-a8bb-405efaf31109 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5584ea3f-8dc3-4830-a8bb-405efaf31109 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:33.960588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:33.961821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:33.962878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 653/1290] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:13:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:33.968285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afee0e-2bc6-48c0-bb00-a01c80a2989b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:34.079480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34afee0e-2bc6-48c0-bb00-a01c80a2989b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:34.112802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdfa7f-c4c2-4ec3-8dd1-cac609c1942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:34.113401 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdfa7f-c4c2-4ec3-8dd1-cac609c1942e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:34.113682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdfa7f-c4c2-4ec3-8dd1-cac609c1942e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:34.113954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdfa7f-c4c2-4ec3-8dd1-cac609c1942e None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:34.114578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdfa7f-c4c2-4ec3-8dd1-cac609c1942e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:34.114880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdfa7f-c4c2-4ec3-8dd1-cac609c1942e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:34.116270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdfa7f-c4c2-4ec3-8dd1-cac609c1942e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:34.121788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 638/1291] 10.222.0.75 () {56 vars in 1116 bytes} [Fri Aug 19 15:13:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:34.127590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584ea3f-8dc3-4830-a8bb-405efaf31109 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:34.230287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584ea3f-8dc3-4830-a8bb-405efaf31109 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:34.262726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca9d0f-ea74-4b57-ad28-1e785f99799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:34.263318 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9d0f-ea74-4b57-ad28-1e785f99799c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:34.263591 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9d0f-ea74-4b57-ad28-1e785f99799c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:34.263859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9d0f-ea74-4b57-ad28-1e785f99799c None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:34.264510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca9d0f-ea74-4b57-ad28-1e785f99799c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:34.264809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca9d0f-ea74-4b57-ad28-1e785f99799c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:34.266167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca9d0f-ea74-4b57-ad28-1e785f99799c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:34.270485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 654/1292] 10.222.0.75 () {56 vars in 1116 bytes} [Fri Aug 19 15:13:34 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:36.161693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe289e9-26cf-431a-9dd9-2ca4116742db None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:36.162256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe289e9-26cf-431a-9dd9-2ca4116742db None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:36.162256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe289e9-26cf-431a-9dd9-2ca4116742db None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:36.163550 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 639/1293] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:36.168652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cf229-e535-4817-ae5d-679758915049 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:36.169073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cf229-e535-4817-ae5d-679758915049 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:36.169215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cf229-e535-4817-ae5d-679758915049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:36.187627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-236cf229-e535-4817-ae5d-679758915049 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:36.188845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-236cf229-e535-4817-ae5d-679758915049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:36.218134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:36.255605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 655/1294] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:36.527059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe289e9-26cf-431a-9dd9-2ca4116742db None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:36.559343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:36.629972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe289e9-26cf-431a-9dd9-2ca4116742db None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:36.663166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faa137-1e40-4aca-b47e-4de1252a1464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:36.663749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faa137-1e40-4aca-b47e-4de1252a1464 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:36.664024 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faa137-1e40-4aca-b47e-4de1252a1464 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:36.664290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faa137-1e40-4aca-b47e-4de1252a1464 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:36.664961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faa137-1e40-4aca-b47e-4de1252a1464 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:36.666322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faa137-1e40-4aca-b47e-4de1252a1464 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:36.684419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 640/1295] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Aug 19 15:13:36 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:13:38.594158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b3d4d-80fd-415d-a17f-49f295275524 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:38.594693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b3d4d-80fd-415d-a17f-49f295275524 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:38.594693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b3d4d-80fd-415d-a17f-49f295275524 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:38.596043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 656/1296] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:38.601155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55357c03-f562-455c-824c-605a40523434 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:38.601593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55357c03-f562-455c-824c-605a40523434 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:38.601697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55357c03-f562-455c-824c-605a40523434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:38.620451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-55357c03-f562-455c-824c-605a40523434 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:38.621803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-55357c03-f562-455c-824c-605a40523434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:38.692594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 641/1297] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:38.956998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b3d4d-80fd-415d-a17f-49f295275524 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:39.077593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b3d4d-80fd-415d-a17f-49f295275524 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:39.113419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570183e-68e5-4aaf-a04e-a0c503c482dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:39.114045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570183e-68e5-4aaf-a04e-a0c503c482dd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:39.114327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570183e-68e5-4aaf-a04e-a0c503c482dd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:39.114598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570183e-68e5-4aaf-a04e-a0c503c482dd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:39.115181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570183e-68e5-4aaf-a04e-a0c503c482dd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:39.116592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570183e-68e5-4aaf-a04e-a0c503c482dd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d570183e-68e5-4aaf-a04e-a0c503c482dd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:39.125140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:39.126157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:39.127119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 657/1298] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:13:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:39.132406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55357c03-f562-455c-824c-605a40523434 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:39.150094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:39.231121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55357c03-f562-455c-824c-605a40523434 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:39.267654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc464b-ac2e-47a8-a74f-2d5e968c5385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:39.268287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc464b-ac2e-47a8-a74f-2d5e968c5385 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:39.268591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc464b-ac2e-47a8-a74f-2d5e968c5385 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:39.268888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc464b-ac2e-47a8-a74f-2d5e968c5385 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:39.269515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc464b-ac2e-47a8-a74f-2d5e968c5385 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:39.269810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc464b-ac2e-47a8-a74f-2d5e968c5385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:39.271170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc464b-ac2e-47a8-a74f-2d5e968c5385 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:39.276952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 642/1299] 10.222.0.75 () {56 vars in 1116 bytes} [Fri Aug 19 15:13:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:39.283161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570183e-68e5-4aaf-a04e-a0c503c482dd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:39.315094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:39.382625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570183e-68e5-4aaf-a04e-a0c503c482dd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:39.416259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0985772e-4206-4ab8-af02-7c1bca8c833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:39.417102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985772e-4206-4ab8-af02-7c1bca8c833b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:39.417391 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985772e-4206-4ab8-af02-7c1bca8c833b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:39.417669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0985772e-4206-4ab8-af02-7c1bca8c833b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:39.418369 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0985772e-4206-4ab8-af02-7c1bca8c833b None admin] RBAC: Adding query filter params (interface=public, service_id=77c4d7edfd5e45cdb98b89d0e15e202a, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:39.418679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0985772e-4206-4ab8-af02-7c1bca8c833b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:39.420064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0985772e-4206-4ab8-af02-7c1bca8c833b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:39.425143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 658/1300] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Aug 19 15:13:39 2022] GET /identity/v3/endpoints?service_id=77c4d7edfd5e45cdb98b89d0e15e202a&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:39.430939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc464b-ac2e-47a8-a74f-2d5e968c5385 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:39.514574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:39.534985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc464b-ac2e-47a8-a74f-2d5e968c5385 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:39.567587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ebe88-14e8-4608-8ca8-5fa4e11f7c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:39.568180 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ebe88-14e8-4608-8ca8-5fa4e11f7c5c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:39.568485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ebe88-14e8-4608-8ca8-5fa4e11f7c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:39.568759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ebe88-14e8-4608-8ca8-5fa4e11f7c5c None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:39.569365 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ebe88-14e8-4608-8ca8-5fa4e11f7c5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:39.570733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ebe88-14e8-4608-8ca8-5fa4e11f7c5c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:39.578077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 643/1301] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Aug 19 15:13:39 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:13:41.457437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f5e64-a914-428f-9643-7d7a5fed4137 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:41.458003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f5e64-a914-428f-9643-7d7a5fed4137 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:41.458003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f5e64-a914-428f-9643-7d7a5fed4137 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:41.459288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 659/1302] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:13:41.464304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a092c-72e8-4529-8492-cefa50d4a5ec None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:41.464749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a092c-72e8-4529-8492-cefa50d4a5ec None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:41.464881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a092c-72e8-4529-8492-cefa50d4a5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:41.483492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-548a092c-72e8-4529-8492-cefa50d4a5ec None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:13:41.484714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-548a092c-72e8-4529-8492-cefa50d4a5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:13:41.550509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 644/1303] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:13:41 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 15:13:41.815928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f5e64-a914-428f-9643-7d7a5fed4137 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:41.912068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:41.919256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f5e64-a914-428f-9643-7d7a5fed4137 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:41.952647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1762fc-3b1d-4934-aa81-1f20ec70e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:41.953264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1762fc-3b1d-4934-aa81-1f20ec70e26d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:41.953542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1762fc-3b1d-4934-aa81-1f20ec70e26d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:41.953812 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1762fc-3b1d-4934-aa81-1f20ec70e26d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:41.954388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1762fc-3b1d-4934-aa81-1f20ec70e26d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:41.955764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1762fc-3b1d-4934-aa81-1f20ec70e26d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0e1762fc-3b1d-4934-aa81-1f20ec70e26d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 15:13:41.964332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 15:13:41.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 19 15:13:41.966410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 660/1304] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Aug 19 15:13:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 15:13:41.971451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a092c-72e8-4529-8492-cefa50d4a5ec None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:42.046346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:42.068766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a092c-72e8-4529-8492-cefa50d4a5ec None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:42.101307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b1b74-2d50-4cc8-8156-e85b872f1199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:42.101897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b1b74-2d50-4cc8-8156-e85b872f1199 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:42.102173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b1b74-2d50-4cc8-8156-e85b872f1199 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:42.102443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b1b74-2d50-4cc8-8156-e85b872f1199 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:42.103058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b1b74-2d50-4cc8-8156-e85b872f1199 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:13:42.103356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b1b74-2d50-4cc8-8156-e85b872f1199 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:42.104866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b1b74-2d50-4cc8-8156-e85b872f1199 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:42.109765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 645/1305] 10.222.0.75 () {56 vars in 1116 bytes} [Fri Aug 19 15:13:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:13:42.115629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1762fc-3b1d-4934-aa81-1f20ec70e26d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:13:42.213697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1762fc-3b1d-4934-aa81-1f20ec70e26d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:13:42.246953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ec5f8-1b36-48ff-8c94-9db0a74fa189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:13:42.247542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ec5f8-1b36-48ff-8c94-9db0a74fa189 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:13:42.247818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ec5f8-1b36-48ff-8c94-9db0a74fa189 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:13:42.248086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ec5f8-1b36-48ff-8c94-9db0a74fa189 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:13:42.248797 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ec5f8-1b36-48ff-8c94-9db0a74fa189 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:13:42.250201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ec5f8-1b36-48ff-8c94-9db0a74fa189 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:13:42.269439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 661/1306] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:13:42 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:13:52.106885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=30 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:13:52.264950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:14:46.484926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0357b83-5869-4638-b1af-63b3d10a6692 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:46.485981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0357b83-5869-4638-b1af-63b3d10a6692 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:46.486477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0357b83-5869-4638-b1af-63b3d10a6692 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:46.492221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 646/1307] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:14:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:14:46.493695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2761f6-a4c3-4d1b-8a6b-54e0efb2593b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:46.494135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2761f6-a4c3-4d1b-8a6b-54e0efb2593b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:46.494306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2761f6-a4c3-4d1b-8a6b-54e0efb2593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:46.586332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4f2761f6-a4c3-4d1b-8a6b-54e0efb2593b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:14:46.587934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4f2761f6-a4c3-4d1b-8a6b-54e0efb2593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:46.711491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 662/1308] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:14:47.019191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eded245-2674-4e57-b9cc-7a91c88d30c1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:47.020027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eded245-2674-4e57-b9cc-7a91c88d30c1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:47.020382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eded245-2674-4e57-b9cc-7a91c88d30c1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:47.022303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 647/1309] 10.222.0.75 () {54 vars in 965 bytes} [Fri Aug 19 15:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:14:47.031389 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b52042-6451-42d7-863b-c2d61f291aff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:47.031681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b52042-6451-42d7-863b-c2d61f291aff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:47.032088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b52042-6451-42d7-863b-c2d61f291aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:47.113883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-14b52042-6451-42d7-863b-c2d61f291aff None None] MFA Rules not processed for user `c8704aae85084d41bdedb5537ef96e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:14:47.115621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-14b52042-6451-42d7-863b-c2d61f291aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:47.136843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:14:47.198070 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 663/1310] 10.222.0.75 () {58 vars in 1062 bytes} [Fri Aug 19 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:14:47.207150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eded245-2674-4e57-b9cc-7a91c88d30c1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:14:47.208991 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7eded245-2674-4e57-b9cc-7a91c88d30c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:47.337465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eded245-2674-4e57-b9cc-7a91c88d30c1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:14:47.423687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005b70a-65cf-4010-80a8-23b2a573b3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:14:47.424300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005b70a-65cf-4010-80a8-23b2a573b3fb service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:47.424607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005b70a-65cf-4010-80a8-23b2a573b3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:47.424876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005b70a-65cf-4010-80a8-23b2a573b3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:47.426521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c005b70a-65cf-4010-80a8-23b2a573b3fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:47.489157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:14:47.551529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005b70a-65cf-4010-80a8-23b2a573b3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:14:47.553019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005b70a-65cf-4010-80a8-23b2a573b3fb service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:14:47.572366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 648/1311] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:14:49.809071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef4d66-9829-4648-8695-3d39c4522ef0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:49.809623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef4d66-9829-4648-8695-3d39c4522ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:49.809623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef4d66-9829-4648-8695-3d39c4522ef0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:49.811925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 664/1312] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:14:49.817503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f72f9c-39fc-438c-a07b-a9318d48ea25 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:49.818054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f72f9c-39fc-438c-a07b-a9318d48ea25 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:49.818054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f72f9c-39fc-438c-a07b-a9318d48ea25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:49.919339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b5f72f9c-39fc-438c-a07b-a9318d48ea25 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:14:49.921417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b5f72f9c-39fc-438c-a07b-a9318d48ea25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:50.035493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 649/1313] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:14:50.321705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef4d66-9829-4648-8695-3d39c4522ef0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:14:50.323324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-edef4d66-9829-4648-8695-3d39c4522ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:50.428110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef4d66-9829-4648-8695-3d39c4522ef0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:14:50.503355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:14:50.505985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f1c40-a871-4d07-b6bb-62ae508e7480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:14:50.506592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1c40-a871-4d07-b6bb-62ae508e7480 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:50.506873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1c40-a871-4d07-b6bb-62ae508e7480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:50.507387 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1c40-a871-4d07-b6bb-62ae508e7480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:50.509412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-810f1c40-a871-4d07-b6bb-62ae508e7480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:50.622316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f1c40-a871-4d07-b6bb-62ae508e7480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:14:50.624577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f1c40-a871-4d07-b6bb-62ae508e7480 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:14:50.643924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 665/1314] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:14:52.866206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5040d-f781-48a8-8e0f-55c7b284b669 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:52.866788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5040d-f781-48a8-8e0f-55c7b284b669 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:52.866788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5040d-f781-48a8-8e0f-55c7b284b669 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:52.868087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 650/1315] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:14:52.873716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bd11c-3fac-460a-82c1-86aabc7c693b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:52.874135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bd11c-3fac-460a-82c1-86aabc7c693b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:52.874340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bd11c-3fac-460a-82c1-86aabc7c693b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:52.940117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-870bd11c-3fac-460a-82c1-86aabc7c693b None None] MFA Rules not processed for user `0c47d47164cb4842b0a2341a28ea82f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:14:52.941290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-870bd11c-3fac-460a-82c1-86aabc7c693b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:52.993533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:14:53.040447 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 666/1316] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:14:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:14:53.327564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5040d-f781-48a8-8e0f-55c7b284b669 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:14:53.329119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bce5040d-f781-48a8-8e0f-55c7b284b669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:53.338750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:14:53.433596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5040d-f781-48a8-8e0f-55c7b284b669 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:14:53.511107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04fa8d-d6c8-414a-bc4a-b8d75989241a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:14:53.511712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04fa8d-d6c8-414a-bc4a-b8d75989241a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:53.511992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04fa8d-d6c8-414a-bc4a-b8d75989241a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:53.512265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04fa8d-d6c8-414a-bc4a-b8d75989241a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:53.513911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8e04fa8d-d6c8-414a-bc4a-b8d75989241a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:53.576444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:14:53.626201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04fa8d-d6c8-414a-bc4a-b8d75989241a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:14:53.627666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04fa8d-d6c8-414a-bc4a-b8d75989241a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:14:53.645488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 651/1317] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:14:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:14:56.135811 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870abf4-2073-4dcb-978b-7fe27f7cfaaa None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:56.136428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870abf4-2073-4dcb-978b-7fe27f7cfaaa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:56.136428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870abf4-2073-4dcb-978b-7fe27f7cfaaa None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:56.142020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 667/1318] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:14:56.143643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89528fa1-a348-408a-87be-e7c0b3a9248e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:56.144089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89528fa1-a348-408a-87be-e7c0b3a9248e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:56.144206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89528fa1-a348-408a-87be-e7c0b3a9248e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:56.219901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-89528fa1-a348-408a-87be-e7c0b3a9248e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:14:56.222325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89528fa1-a348-408a-87be-e7c0b3a9248e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:56.328699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 652/1319] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:14:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:14:56.632197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435c6ea-8d1a-4269-b708-48f7cd74c52e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:56.632558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435c6ea-8d1a-4269-b708-48f7cd74c52e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:56.632829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435c6ea-8d1a-4269-b708-48f7cd74c52e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:56.634151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 668/1320] 10.222.0.75 () {54 vars in 965 bytes} [Fri Aug 19 15:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:14:56.641005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70019bf3-64db-4260-9850-62f9768ecb4f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:56.641329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70019bf3-64db-4260-9850-62f9768ecb4f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:56.641602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70019bf3-64db-4260-9850-62f9768ecb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:56.663266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:14:56.719322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-70019bf3-64db-4260-9850-62f9768ecb4f None None] MFA Rules not processed for user `c8704aae85084d41bdedb5537ef96e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:14:56.720564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-70019bf3-64db-4260-9850-62f9768ecb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:56.797746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 653/1321] 10.222.0.75 () {58 vars in 1062 bytes} [Fri Aug 19 15:14:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:14:56.806688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435c6ea-8d1a-4269-b708-48f7cd74c52e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:14:56.808276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7435c6ea-8d1a-4269-b708-48f7cd74c52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:56.911309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435c6ea-8d1a-4269-b708-48f7cd74c52e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:14:56.984607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4f772-5422-4231-9acb-282ee2b8d560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:14:56.985211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4f772-5422-4231-9acb-282ee2b8d560 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:14:56.985491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4f772-5422-4231-9acb-282ee2b8d560 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:14:56.985764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4f772-5422-4231-9acb-282ee2b8d560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:14:56.987368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-87f4f772-5422-4231-9acb-282ee2b8d560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:14:56.995372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:14:57.099790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4f772-5422-4231-9acb-282ee2b8d560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:14:57.101274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4f772-5422-4231-9acb-282ee2b8d560 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:14:57.118897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 669/1322] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:14:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:00.188041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f96439-2ba3-4cf2-aae3-fb7756e9038a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:00.188607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f96439-2ba3-4cf2-aae3-fb7756e9038a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:00.188607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f96439-2ba3-4cf2-aae3-fb7756e9038a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:00.190015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 654/1323] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:00.195296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039cede-7fe3-42fd-85b6-bc78df025f10 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:00.195714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039cede-7fe3-42fd-85b6-bc78df025f10 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:00.195855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039cede-7fe3-42fd-85b6-bc78df025f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:00.263388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7039cede-7fe3-42fd-85b6-bc78df025f10 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:00.264779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7039cede-7fe3-42fd-85b6-bc78df025f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:00.324375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:00.365836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 670/1324] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:00.657846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f96439-2ba3-4cf2-aae3-fb7756e9038a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:00.659705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-19f96439-2ba3-4cf2-aae3-fb7756e9038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:00.772350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f96439-2ba3-4cf2-aae3-fb7756e9038a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:00.786166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:00.849926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da54e1-4d28-4b38-9680-487a081bfc76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:00.850527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da54e1-4d28-4b38-9680-487a081bfc76 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:00.850800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da54e1-4d28-4b38-9680-487a081bfc76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:00.851069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da54e1-4d28-4b38-9680-487a081bfc76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:00.852694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-15da54e1-4d28-4b38-9680-487a081bfc76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:00.964469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da54e1-4d28-4b38-9680-487a081bfc76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:00.965797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da54e1-4d28-4b38-9680-487a081bfc76 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:00.984236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 655/1325] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:03.001104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8318861-7f1c-4528-9d94-b7e56f8ece7f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:03.001721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8318861-7f1c-4528-9d94-b7e56f8ece7f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:03.001721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8318861-7f1c-4528-9d94-b7e56f8ece7f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:03.002983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 671/1326] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:03.008876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04b0ab-b910-47ce-bbad-8be1203f8284 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:03.010216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04b0ab-b910-47ce-bbad-8be1203f8284 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:03.010577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04b0ab-b910-47ce-bbad-8be1203f8284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:03.060543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:03.080629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1f04b0ab-b910-47ce-bbad-8be1203f8284 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:03.081830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1f04b0ab-b910-47ce-bbad-8be1203f8284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:03.182583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 656/1327] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:03.465038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8318861-7f1c-4528-9d94-b7e56f8ece7f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:03.466598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c8318861-7f1c-4528-9d94-b7e56f8ece7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:03.571055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8318861-7f1c-4528-9d94-b7e56f8ece7f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:03.652254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb124f7f-fd6f-428a-bd43-28f70490f25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:03.652888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb124f7f-fd6f-428a-bd43-28f70490f25c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:03.653196 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb124f7f-fd6f-428a-bd43-28f70490f25c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:03.653474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb124f7f-fd6f-428a-bd43-28f70490f25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:03.655750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eb124f7f-fd6f-428a-bd43-28f70490f25c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:03.677885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:03.773082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb124f7f-fd6f-428a-bd43-28f70490f25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:03.774544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb124f7f-fd6f-428a-bd43-28f70490f25c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:03.793496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 672/1328] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:05.799369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe66da3-acb2-4fff-b8c3-1bf4a9333626 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:05.799868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe66da3-acb2-4fff-b8c3-1bf4a9333626 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:05.799964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe66da3-acb2-4fff-b8c3-1bf4a9333626 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:05.801327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 657/1329] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:05.806639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f072139-0e69-4cdf-b0b1-81fa59d5c40b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:05.807062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f072139-0e69-4cdf-b0b1-81fa59d5c40b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:05.807204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f072139-0e69-4cdf-b0b1-81fa59d5c40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:05.882292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1f072139-0e69-4cdf-b0b1-81fa59d5c40b None None] MFA Rules not processed for user `0c47d47164cb4842b0a2341a28ea82f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:05.883514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f072139-0e69-4cdf-b0b1-81fa59d5c40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:05.948857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:05.980554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 673/1330] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:05 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:06.274759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe66da3-acb2-4fff-b8c3-1bf4a9333626 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:06.276290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dbe66da3-acb2-4fff-b8c3-1bf4a9333626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:06.392360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe66da3-acb2-4fff-b8c3-1bf4a9333626 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:06.413087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:06.465759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298cde9-39de-4845-89f5-b103628a3b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:06.466355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298cde9-39de-4845-89f5-b103628a3b07 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:06.466629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298cde9-39de-4845-89f5-b103628a3b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:06.467404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298cde9-39de-4845-89f5-b103628a3b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:06.469135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a298cde9-39de-4845-89f5-b103628a3b07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:06.580057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298cde9-39de-4845-89f5-b103628a3b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:06.581582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298cde9-39de-4845-89f5-b103628a3b07 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:06.602595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 658/1331] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:08.765427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588074c-3455-4a55-ab76-91de20e1f622 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:08.766003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588074c-3455-4a55-ab76-91de20e1f622 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:08.766003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588074c-3455-4a55-ab76-91de20e1f622 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:08.767308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 674/1332] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:08.772787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c20a7f-9ffe-441e-a10a-d8c0b0237141 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:08.773220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c20a7f-9ffe-441e-a10a-d8c0b0237141 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:08.773341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c20a7f-9ffe-441e-a10a-d8c0b0237141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:08.838380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:08.841070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f6c20a7f-9ffe-441e-a10a-d8c0b0237141 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:08.843350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f6c20a7f-9ffe-441e-a10a-d8c0b0237141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:08.939763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 659/1333] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:09.220419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588074c-3455-4a55-ab76-91de20e1f622 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:09.221979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d588074c-3455-4a55-ab76-91de20e1f622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:09.327283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588074c-3455-4a55-ab76-91de20e1f622 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:09.400874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df981fd0-0faf-4cdc-9ce8-85614d5fa353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:09.401470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df981fd0-0faf-4cdc-9ce8-85614d5fa353 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:09.401743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df981fd0-0faf-4cdc-9ce8-85614d5fa353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:09.402009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df981fd0-0faf-4cdc-9ce8-85614d5fa353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:09.403596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-df981fd0-0faf-4cdc-9ce8-85614d5fa353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:09.429609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:09.513492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df981fd0-0faf-4cdc-9ce8-85614d5fa353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:09.515571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df981fd0-0faf-4cdc-9ce8-85614d5fa353 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:09.533284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 675/1334] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:11.509561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb1bac-14c3-4fdd-87da-8e82de852777 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:11.510137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb1bac-14c3-4fdd-87da-8e82de852777 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:11.510137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb1bac-14c3-4fdd-87da-8e82de852777 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:11.511460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 660/1335] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:11.517289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00035b65-54d1-41e2-a317-49fedf0ca844 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:11.517693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00035b65-54d1-41e2-a317-49fedf0ca844 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:11.517834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00035b65-54d1-41e2-a317-49fedf0ca844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:11.584324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-00035b65-54d1-41e2-a317-49fedf0ca844 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:11.585519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-00035b65-54d1-41e2-a317-49fedf0ca844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:11.662249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:11.685240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 676/1336] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:11.987280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb1bac-14c3-4fdd-87da-8e82de852777 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:11.988964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5beb1bac-14c3-4fdd-87da-8e82de852777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:12.092293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb1bac-14c3-4fdd-87da-8e82de852777 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:12.120312 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:12.165155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa315-8dba-407b-ab20-34bddd579811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:12.165757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa315-8dba-407b-ab20-34bddd579811 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:12.166077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa315-8dba-407b-ab20-34bddd579811 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:12.166333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa315-8dba-407b-ab20-34bddd579811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:12.167913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-df5fa315-8dba-407b-ab20-34bddd579811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:12.277712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa315-8dba-407b-ab20-34bddd579811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:12.279170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa315-8dba-407b-ab20-34bddd579811 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:12.298524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 661/1337] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:14.648612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdaba3-82bd-4ef4-9f9b-eeb732068180 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:14.649150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdaba3-82bd-4ef4-9f9b-eeb732068180 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:14.649150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdaba3-82bd-4ef4-9f9b-eeb732068180 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:14.650490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 677/1338] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:14.655598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d799a8-0564-44dc-a13c-ac4a1230870e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:14.656027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d799a8-0564-44dc-a13c-ac4a1230870e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:14.656146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d799a8-0564-44dc-a13c-ac4a1230870e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:14.723506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-66d799a8-0564-44dc-a13c-ac4a1230870e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:14.724736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-66d799a8-0564-44dc-a13c-ac4a1230870e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:14.733061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:14.822321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 662/1339] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:15.105230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdaba3-82bd-4ef4-9f9b-eeb732068180 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:15.106805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e8cdaba3-82bd-4ef4-9f9b-eeb732068180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:15.213802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdaba3-82bd-4ef4-9f9b-eeb732068180 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:15.287613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1370f6-8f52-4759-af10-6f9738087b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:15.288215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1370f6-8f52-4759-af10-6f9738087b81 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:15.288523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1370f6-8f52-4759-af10-6f9738087b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:15.288796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1370f6-8f52-4759-af10-6f9738087b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:15.290400 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e1370f6-8f52-4759-af10-6f9738087b81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:15.326547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:15.399480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1370f6-8f52-4759-af10-6f9738087b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:15.400915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1370f6-8f52-4759-af10-6f9738087b81 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:15.420618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 678/1340] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:19.273398 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cc8a8-9482-4983-8122-9e492a65e130 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:19.273966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cc8a8-9482-4983-8122-9e492a65e130 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:19.273966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cc8a8-9482-4983-8122-9e492a65e130 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:19.275273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 663/1341] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:19.280880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0eb60-3b7a-4b58-995b-4b7db78bb6d4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:19.281284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0eb60-3b7a-4b58-995b-4b7db78bb6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:19.281466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0eb60-3b7a-4b58-995b-4b7db78bb6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:19.351070 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bcd0eb60-3b7a-4b58-995b-4b7db78bb6d4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:19.352267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bcd0eb60-3b7a-4b58-995b-4b7db78bb6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:19.442876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:19.455280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 679/1342] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:19.737844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cc8a8-9482-4983-8122-9e492a65e130 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:19.739369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-090cc8a8-9482-4983-8122-9e492a65e130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:19.849236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cc8a8-9482-4983-8122-9e492a65e130 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:19.888580 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:19.926155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617be1b-c714-4611-b873-d4d4b6c32088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:19.926736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617be1b-c714-4611-b873-d4d4b6c32088 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:19.927057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617be1b-c714-4611-b873-d4d4b6c32088 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:19.927358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617be1b-c714-4611-b873-d4d4b6c32088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:19.929141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c617be1b-c714-4611-b873-d4d4b6c32088 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:20.046350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617be1b-c714-4611-b873-d4d4b6c32088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:20.047831 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617be1b-c714-4611-b873-d4d4b6c32088 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:20.066416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 664/1343] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:22.629960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93b53c-c8d0-41cd-ac17-e2a93fed5e53 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:22.630509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93b53c-c8d0-41cd-ac17-e2a93fed5e53 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:22.630626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93b53c-c8d0-41cd-ac17-e2a93fed5e53 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:22.631938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 680/1344] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:22.637422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f650f-c545-40fe-878a-d4601563341e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:22.638265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f650f-c545-40fe-878a-d4601563341e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:22.638544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f650f-c545-40fe-878a-d4601563341e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:22.715437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-205f650f-c545-40fe-878a-d4601563341e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:22.716682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-205f650f-c545-40fe-878a-d4601563341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:22.736613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:22.819981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 665/1345] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:23.114565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93b53c-c8d0-41cd-ac17-e2a93fed5e53 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:23.116097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e93b53c-c8d0-41cd-ac17-e2a93fed5e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:23.223465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93b53c-c8d0-41cd-ac17-e2a93fed5e53 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:23.298810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc77efe-c17f-4380-a400-b596f8d71f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:23.299420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc77efe-c17f-4380-a400-b596f8d71f38 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:23.299700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc77efe-c17f-4380-a400-b596f8d71f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:23.299973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc77efe-c17f-4380-a400-b596f8d71f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:23.301599 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8cc77efe-c17f-4380-a400-b596f8d71f38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:23.349229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:23.419751 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc77efe-c17f-4380-a400-b596f8d71f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:23.421375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc77efe-c17f-4380-a400-b596f8d71f38 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:23.440369 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 681/1346] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:27.582862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b71be0-d3da-448d-bad5-cb2766287c74 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:27.583408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b71be0-d3da-448d-bad5-cb2766287c74 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:27.583408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b71be0-d3da-448d-bad5-cb2766287c74 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:27.586244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 666/1347] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:27.591771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472943-a6af-4008-992b-fc2912f3691f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:27.592348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472943-a6af-4008-992b-fc2912f3691f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:27.592492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472943-a6af-4008-992b-fc2912f3691f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:27.665457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-62472943-a6af-4008-992b-fc2912f3691f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:27.666696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-62472943-a6af-4008-992b-fc2912f3691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:27.771314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:27.774196 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 682/1348] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:28.058019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b71be0-d3da-448d-bad5-cb2766287c74 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:28.059533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-22b71be0-d3da-448d-bad5-cb2766287c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:28.169918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b71be0-d3da-448d-bad5-cb2766287c74 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:28.218962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:28.251222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daef16-0c07-47bd-8041-e89ad083db3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:28.251833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daef16-0c07-47bd-8041-e89ad083db3b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:28.252108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daef16-0c07-47bd-8041-e89ad083db3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:28.252371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daef16-0c07-47bd-8041-e89ad083db3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:28.254045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d9daef16-0c07-47bd-8041-e89ad083db3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:28.375844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daef16-0c07-47bd-8041-e89ad083db3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:28.377231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daef16-0c07-47bd-8041-e89ad083db3b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:28.396172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 667/1349] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:33.437160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a4d51-67c6-4674-8b1c-04888253d594 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:33.437712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a4d51-67c6-4674-8b1c-04888253d594 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:33.437712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a4d51-67c6-4674-8b1c-04888253d594 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:33.439279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 683/1350] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:33.444854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ba34d-6fab-4288-b018-c75db42560f5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:33.445349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ba34d-6fab-4288-b018-c75db42560f5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:33.445592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ba34d-6fab-4288-b018-c75db42560f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:33.568299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-934ba34d-6fab-4288-b018-c75db42560f5 None None] MFA Rules not processed for user `bc2920aac4c046aca75473158ec204cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:33.573388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-934ba34d-6fab-4288-b018-c75db42560f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:33.627567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:33.750019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 668/1351] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:15:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:15:37.895314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655934b6-61c1-4223-910d-82671cefc503 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:37.895844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655934b6-61c1-4223-910d-82671cefc503 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:37.895844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655934b6-61c1-4223-910d-82671cefc503 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:37.897385 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 684/1352] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:37.902198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a19b0-98d2-4b32-900b-bf31c7bd34a7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:37.902653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a19b0-98d2-4b32-900b-bf31c7bd34a7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:37.902938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a19b0-98d2-4b32-900b-bf31c7bd34a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:37.990193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-946a19b0-98d2-4b32-900b-bf31c7bd34a7 None None] MFA Rules not processed for user `bc2920aac4c046aca75473158ec204cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:37.991654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-946a19b0-98d2-4b32-900b-bf31c7bd34a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:38.071574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:38.111439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 669/1353] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:15:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:39.609487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4fe9-eb48-46a7-9415-bad98ffee64f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:39.610065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4fe9-eb48-46a7-9415-bad98ffee64f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:39.610185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4fe9-eb48-46a7-9415-bad98ffee64f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:39.611570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 685/1354] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:15:39.616853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb6053-4aad-40b5-b1ee-01bfefcda67b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:39.617296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb6053-4aad-40b5-b1ee-01bfefcda67b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:39.617391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb6053-4aad-40b5-b1ee-01bfefcda67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:39.698075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ecbb6053-4aad-40b5-b1ee-01bfefcda67b None None] MFA Rules not processed for user `bc2920aac4c046aca75473158ec204cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:39.699306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ecbb6053-4aad-40b5-b1ee-01bfefcda67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:39.816319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 670/1355] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:15:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:43.117405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955c83c-e8a8-4264-9216-d3a59602481a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:43.117963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955c83c-e8a8-4264-9216-d3a59602481a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:43.117963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955c83c-e8a8-4264-9216-d3a59602481a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:43.119291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 686/1356] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:43.124646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33ae92-91ad-4ed5-a79c-c6716cb41c04 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:43.125061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33ae92-91ad-4ed5-a79c-c6716cb41c04 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:43.125198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33ae92-91ad-4ed5-a79c-c6716cb41c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:43.160733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:43.193130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bc33ae92-91ad-4ed5-a79c-c6716cb41c04 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:43.194334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bc33ae92-91ad-4ed5-a79c-c6716cb41c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:43.302423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 671/1357] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:43.646480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54feed-aae0-4539-aff3-8b6aa726452c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:43.646954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54feed-aae0-4539-aff3-8b6aa726452c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:43.647372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54feed-aae0-4539-aff3-8b6aa726452c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:43.648944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 687/1358] 10.222.0.75 () {54 vars in 962 bytes} [Fri Aug 19 15:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:43.665839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c56daf-3cad-4112-9a2f-d928952c1bef None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:43.666450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c56daf-3cad-4112-9a2f-d928952c1bef None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:43.666960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c56daf-3cad-4112-9a2f-d928952c1bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:43.807827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-07c56daf-3cad-4112-9a2f-d928952c1bef None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:43.809790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-07c56daf-3cad-4112-9a2f-d928952c1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:43.894466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:44.022278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 672/1359] 10.222.0.75 () {58 vars in 1059 bytes} [Fri Aug 19 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:44.031831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54feed-aae0-4539-aff3-8b6aa726452c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:44.033357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec54feed-aae0-4539-aff3-8b6aa726452c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:44.213702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54feed-aae0-4539-aff3-8b6aa726452c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:44.333258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:44.400598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69952d-560b-4ec8-80c0-a2bd3804b57a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:44.401495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69952d-560b-4ec8-80c0-a2bd3804b57a service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:44.401900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69952d-560b-4ec8-80c0-a2bd3804b57a service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:44.402310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69952d-560b-4ec8-80c0-a2bd3804b57a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:44.405652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d69952d-560b-4ec8-80c0-a2bd3804b57a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:44.539932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69952d-560b-4ec8-80c0-a2bd3804b57a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:44.541850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69952d-560b-4ec8-80c0-a2bd3804b57a service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:44.565507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 688/1360] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:47.118321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094487a-28da-4ffb-b5a2-caa88ab96422 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:47.118911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094487a-28da-4ffb-b5a2-caa88ab96422 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:47.118911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094487a-28da-4ffb-b5a2-caa88ab96422 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:47.120358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 673/1361] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:47.126321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15048670-dfdd-42b7-80e7-0bb4790ca0cc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:47.126925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15048670-dfdd-42b7-80e7-0bb4790ca0cc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:47.126925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15048670-dfdd-42b7-80e7-0bb4790ca0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:47.203428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-15048670-dfdd-42b7-80e7-0bb4790ca0cc None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:47.204720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-15048670-dfdd-42b7-80e7-0bb4790ca0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:47.247352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:47.320306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 689/1362] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:47.612170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094487a-28da-4ffb-b5a2-caa88ab96422 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:47.613701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a094487a-28da-4ffb-b5a2-caa88ab96422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:47.808336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094487a-28da-4ffb-b5a2-caa88ab96422 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:47.821079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:47.952165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65938e32-36f1-4ba7-8b1a-0ee6d2243100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:47.953597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65938e32-36f1-4ba7-8b1a-0ee6d2243100 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:47.953911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65938e32-36f1-4ba7-8b1a-0ee6d2243100 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:47.954380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65938e32-36f1-4ba7-8b1a-0ee6d2243100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:47.956118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-65938e32-36f1-4ba7-8b1a-0ee6d2243100 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:48.068194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65938e32-36f1-4ba7-8b1a-0ee6d2243100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:48.070064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65938e32-36f1-4ba7-8b1a-0ee6d2243100 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:48.093669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 674/1363] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:15:50.604355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb81d4-0aa9-4a73-9c68-508601ca08a1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:50.605236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb81d4-0aa9-4a73-9c68-508601ca08a1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:50.605236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb81d4-0aa9-4a73-9c68-508601ca08a1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:50.607000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 690/1364] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:15:50.613793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea5feb-917e-429c-988b-ab56f739c8fa None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:50.614579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea5feb-917e-429c-988b-ab56f739c8fa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:50.614765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea5feb-917e-429c-988b-ab56f739c8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:50.633695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:50.710344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-01ea5feb-917e-429c-988b-ab56f739c8fa None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:50.711791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-01ea5feb-917e-429c-988b-ab56f739c8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:50.831904 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 675/1365] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:15:51.333746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92179e-0289-43e1-8a3d-766323cd6b3c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:51.334032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92179e-0289-43e1-8a3d-766323cd6b3c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:51.334304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92179e-0289-43e1-8a3d-766323cd6b3c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:51.335787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 691/1366] 10.222.0.75 () {54 vars in 962 bytes} [Fri Aug 19 15:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:51.344913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92804d16-1d33-415d-8c11-6e419b065898 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:51.345323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92804d16-1d33-415d-8c11-6e419b065898 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:51.345788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92804d16-1d33-415d-8c11-6e419b065898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:51.475011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-92804d16-1d33-415d-8c11-6e419b065898 None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:51.476859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-92804d16-1d33-415d-8c11-6e419b065898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:51.537183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:51.662454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 676/1367] 10.222.0.75 () {58 vars in 1059 bytes} [Fri Aug 19 15:15:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:51.672975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92179e-0289-43e1-8a3d-766323cd6b3c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:51.675012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2f92179e-0289-43e1-8a3d-766323cd6b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:51.887151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92179e-0289-43e1-8a3d-766323cd6b3c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:51.911420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:52.020748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f60b-ad97-4606-93ce-b043f6610af6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:52.021363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f60b-ad97-4606-93ce-b043f6610af6 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:52.021646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f60b-ad97-4606-93ce-b043f6610af6 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:52.021919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f60b-ad97-4606-93ce-b043f6610af6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:52.023557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0bb6f60b-ad97-4606-93ce-b043f6610af6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:52.142324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f60b-ad97-4606-93ce-b043f6610af6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:52.143965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f60b-ad97-4606-93ce-b043f6610af6 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:52.163939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 692/1368] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:52.349723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7e51b-23d9-439a-a4c4-0049b5843f46 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:52.350293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7e51b-23d9-439a-a4c4-0049b5843f46 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:52.350636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7e51b-23d9-439a-a4c4-0049b5843f46 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:52.352103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 677/1369] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:52.362229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860ae3-c06b-4a86-9a9e-e8dd52e8d19a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:52.362718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860ae3-c06b-4a86-9a9e-e8dd52e8d19a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:52.363223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a860ae3-c06b-4a86-9a9e-e8dd52e8d19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:52.429980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:52.493985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4a860ae3-c06b-4a86-9a9e-e8dd52e8d19a None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:52.495527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4a860ae3-c06b-4a86-9a9e-e8dd52e8d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:52.617619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 693/1370] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:53.207310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04ac38-4355-4169-8e39-5360d64527e1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:53.208009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04ac38-4355-4169-8e39-5360d64527e1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:53.208009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04ac38-4355-4169-8e39-5360d64527e1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:53.209466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 678/1371] 10.222.0.75 () {54 vars in 964 bytes} [Fri Aug 19 15:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:15:53.214561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f9ca7-f8e0-4f14-8148-285d8ba473fa None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:53.215442 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f9ca7-f8e0-4f14-8148-285d8ba473fa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:53.215732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f9ca7-f8e0-4f14-8148-285d8ba473fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:53.303457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f09f9ca7-f8e0-4f14-8148-285d8ba473fa None None] MFA Rules not processed for user `a6b4d6a93c7e49009aca527e281b5c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:53.304687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f09f9ca7-f8e0-4f14-8148-285d8ba473fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:53.358904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:53.400330 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 694/1372] 10.222.0.75 () {58 vars in 1061 bytes} [Fri Aug 19 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:53.409087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04ac38-4355-4169-8e39-5360d64527e1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:53.410900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0a04ac38-4355-4169-8e39-5360d64527e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:53.553440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04ac38-4355-4169-8e39-5360d64527e1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:53.607179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:53.649867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb0fec-8792-4e52-8faa-be4389d95d90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:53.650687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb0fec-8792-4e52-8faa-be4389d95d90 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:53.650839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb0fec-8792-4e52-8faa-be4389d95d90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:53.651134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb0fec-8792-4e52-8faa-be4389d95d90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:53.652766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7bdb0fec-8792-4e52-8faa-be4389d95d90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:53.825895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb0fec-8792-4e52-8faa-be4389d95d90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:53.830407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb0fec-8792-4e52-8faa-be4389d95d90 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:53.871207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 679/1373] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:15:54.240470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831e455-8b87-48d3-8b96-7f2594cb53a7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:54.241052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831e455-8b87-48d3-8b96-7f2594cb53a7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:54.241052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831e455-8b87-48d3-8b96-7f2594cb53a7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:54.242345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 695/1374] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:15:54.247572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82883899-5769-4a4b-9295-a63b0b5fa3ca None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:54.248701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82883899-5769-4a4b-9295-a63b0b5fa3ca None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:54.248701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82883899-5769-4a4b-9295-a63b0b5fa3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:54.337214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-82883899-5769-4a4b-9295-a63b0b5fa3ca None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:54.338652 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-82883899-5769-4a4b-9295-a63b0b5fa3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:54.359841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:54.440850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 680/1375] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:15:54.936479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831e455-8b87-48d3-8b96-7f2594cb53a7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:54.938173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5831e455-8b87-48d3-8b96-7f2594cb53a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:55.089160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831e455-8b87-48d3-8b96-7f2594cb53a7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:55.142460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:55.218210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18671347-7c3c-4025-8218-723e1300cb43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:55.218845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18671347-7c3c-4025-8218-723e1300cb43 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:55.219102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18671347-7c3c-4025-8218-723e1300cb43 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:55.219384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18671347-7c3c-4025-8218-723e1300cb43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:55.221046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-18671347-7c3c-4025-8218-723e1300cb43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:55.362737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18671347-7c3c-4025-8218-723e1300cb43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:55.365757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18671347-7c3c-4025-8218-723e1300cb43 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:55.390823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 696/1376] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:15:57.632547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c806a1-022b-47ba-b5c0-733d9aba041c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:57.633264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c806a1-022b-47ba-b5c0-733d9aba041c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:57.633264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c806a1-022b-47ba-b5c0-733d9aba041c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:57.634452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 681/1377] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:15:57.640979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb9d77-b39a-4c10-aaa3-24bcf4c29437 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:57.641492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb9d77-b39a-4c10-aaa3-24bcf4c29437 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:57.644625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb9d77-b39a-4c10-aaa3-24bcf4c29437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:57.741622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0eeb9d77-b39a-4c10-aaa3-24bcf4c29437 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:15:57.742844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0eeb9d77-b39a-4c10-aaa3-24bcf4c29437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:57.751454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:57.899373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 697/1378] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:15:58.197584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c806a1-022b-47ba-b5c0-733d9aba041c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:15:58.199098 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-01c806a1-022b-47ba-b5c0-733d9aba041c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:58.360617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c806a1-022b-47ba-b5c0-733d9aba041c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:15:58.364980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:15:58.491326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfe601-51ec-44ff-8f88-a451e8cc444a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:15:58.492319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfe601-51ec-44ff-8f88-a451e8cc444a service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:15:58.492872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfe601-51ec-44ff-8f88-a451e8cc444a service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:15:58.493351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfe601-51ec-44ff-8f88-a451e8cc444a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:15:58.495120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-44dfe601-51ec-44ff-8f88-a451e8cc444a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:15:58.600177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfe601-51ec-44ff-8f88-a451e8cc444a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:15:58.601985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfe601-51ec-44ff-8f88-a451e8cc444a service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:15:58.622266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 682/1379] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:15:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:00.820701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad7733-d5dd-433e-9e08-6e249b2645d2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:00.821192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad7733-d5dd-433e-9e08-6e249b2645d2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:00.821609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad7733-d5dd-433e-9e08-6e249b2645d2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:00.823917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 698/1380] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:00.830212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1019b4-6e40-4e32-838e-25f07d621842 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:00.830949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1019b4-6e40-4e32-838e-25f07d621842 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:00.830949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1019b4-6e40-4e32-838e-25f07d621842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:00.843258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:00.913996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2f1019b4-6e40-4e32-838e-25f07d621842 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:00.915292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2f1019b4-6e40-4e32-838e-25f07d621842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:01.032040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 683/1381] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:16:01.339603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad7733-d5dd-433e-9e08-6e249b2645d2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:01.341154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b2ad7733-d5dd-433e-9e08-6e249b2645d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:01.503365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:01.516931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad7733-d5dd-433e-9e08-6e249b2645d2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:01.644439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37264fa2-47b2-418b-a44b-4a909ff5a3d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:01.645050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264fa2-47b2-418b-a44b-4a909ff5a3d4 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:01.645328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264fa2-47b2-418b-a44b-4a909ff5a3d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:01.645598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37264fa2-47b2-418b-a44b-4a909ff5a3d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:01.647187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-37264fa2-47b2-418b-a44b-4a909ff5a3d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:01.753752 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37264fa2-47b2-418b-a44b-4a909ff5a3d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:01.755368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37264fa2-47b2-418b-a44b-4a909ff5a3d4 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:01.767890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:01.777422 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 699/1382] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:16:04.055484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba705cbb-2ec7-409a-9ae8-8edc11df60ac None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:04.056069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba705cbb-2ec7-409a-9ae8-8edc11df60ac None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:04.056069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba705cbb-2ec7-409a-9ae8-8edc11df60ac None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:04.057397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 684/1383] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:16:04.066892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5195-8a25-46d1-91f5-bbcb008f0384 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:04.067628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5195-8a25-46d1-91f5-bbcb008f0384 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:04.068379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5195-8a25-46d1-91f5-bbcb008f0384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:04.190240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1eda5195-8a25-46d1-91f5-bbcb008f0384 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:04.192411 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1eda5195-8a25-46d1-91f5-bbcb008f0384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:04.335316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 700/1384] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:04.729817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba705cbb-2ec7-409a-9ae8-8edc11df60ac None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:04.733427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ba705cbb-2ec7-409a-9ae8-8edc11df60ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:04.856364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:04.927461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba705cbb-2ec7-409a-9ae8-8edc11df60ac None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:05.093083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241a2a3-e252-4bf4-854a-654cb7914c07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:05.094215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241a2a3-e252-4bf4-854a-654cb7914c07 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:05.094215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241a2a3-e252-4bf4-854a-654cb7914c07 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:05.094475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241a2a3-e252-4bf4-854a-654cb7914c07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:05.095904 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7241a2a3-e252-4bf4-854a-654cb7914c07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:05.167213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:05.233040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241a2a3-e252-4bf4-854a-654cb7914c07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:05.235284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241a2a3-e252-4bf4-854a-654cb7914c07 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:05.264383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 685/1385] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:07.923771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567f837-1737-4a4c-beb9-d39aec79941e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:07.924767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567f837-1737-4a4c-beb9-d39aec79941e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:07.924767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567f837-1737-4a4c-beb9-d39aec79941e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:07.926345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 701/1386] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:16:07.933238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86b7a3-b49f-4b6a-b676-05e28ec46054 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:07.934032 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86b7a3-b49f-4b6a-b676-05e28ec46054 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:07.934245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86b7a3-b49f-4b6a-b676-05e28ec46054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:08.017245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-af86b7a3-b49f-4b6a-b676-05e28ec46054 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:08.018484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-af86b7a3-b49f-4b6a-b676-05e28ec46054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:08.140048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 686/1387] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:08.444172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567f837-1737-4a4c-beb9-d39aec79941e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:08.445700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5567f837-1737-4a4c-beb9-d39aec79941e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:08.530630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:08.611879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567f837-1737-4a4c-beb9-d39aec79941e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:08.738684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9790f0-aeb6-4dff-b556-c3c3eecdfa95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:08.739298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9790f0-aeb6-4dff-b556-c3c3eecdfa95 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:08.739568 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9790f0-aeb6-4dff-b556-c3c3eecdfa95 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:08.739854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9790f0-aeb6-4dff-b556-c3c3eecdfa95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:08.741562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4d9790f0-aeb6-4dff-b556-c3c3eecdfa95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:08.786036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:08.844287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9790f0-aeb6-4dff-b556-c3c3eecdfa95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:08.846078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9790f0-aeb6-4dff-b556-c3c3eecdfa95 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:08.865207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 702/1388] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:16:10.126972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e92630-12f9-441b-897d-d7e2c8f95bd2 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:10.127628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e92630-12f9-441b-897d-d7e2c8f95bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:10.127628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e92630-12f9-441b-897d-d7e2c8f95bd2 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:10.129049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 687/1389] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:10.134537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6c442-4ae5-4fcb-a8fb-9614b51544f1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:10.134988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6c442-4ae5-4fcb-a8fb-9614b51544f1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:10.135203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6c442-4ae5-4fcb-a8fb-9614b51544f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:10.205985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a3f6c442-4ae5-4fcb-a8fb-9614b51544f1 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:10.207207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3f6c442-4ae5-4fcb-a8fb-9614b51544f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:10.323719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 703/1390] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:16:10.627237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa5838-5ef3-4f0f-88ed-4b391b25e873 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:10.627515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa5838-5ef3-4f0f-88ed-4b391b25e873 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:10.627786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa5838-5ef3-4f0f-88ed-4b391b25e873 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:10.629159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 688/1391] 10.222.0.75 () {54 vars in 967 bytes} [Fri Aug 19 15:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:10.634379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682e3cf-6d7b-4af5-b853-41bc653685e3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:10.634888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682e3cf-6d7b-4af5-b853-41bc653685e3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:10.635267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682e3cf-6d7b-4af5-b853-41bc653685e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:10.652714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:10.721910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3682e3cf-6d7b-4af5-b853-41bc653685e3 None None] MFA Rules not processed for user `de0dfcbad22348fdaea32d64557e68a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:10.723138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3682e3cf-6d7b-4af5-b853-41bc653685e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:10.803170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 704/1392] 10.222.0.75 () {58 vars in 1064 bytes} [Fri Aug 19 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:10.812053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa5838-5ef3-4f0f-88ed-4b391b25e873 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:10.813726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6aaa5838-5ef3-4f0f-88ed-4b391b25e873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:10.829791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fd59-1447-416d-95f3-63486b3f6911 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:10.830075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fd59-1447-416d-95f3-63486b3f6911 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:10.830352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fd59-1447-416d-95f3-63486b3f6911 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:10.831847 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 705/1393] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:16:10.836665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a011520-a4a9-422d-a7b8-c7748a31de3c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:10.836945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a011520-a4a9-422d-a7b8-c7748a31de3c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:10.837218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a011520-a4a9-422d-a7b8-c7748a31de3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:10.847488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:10.911056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3a011520-a4a9-422d-a7b8-c7748a31de3c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:10.912284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3a011520-a4a9-422d-a7b8-c7748a31de3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:10.941365 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa5838-5ef3-4f0f-88ed-4b391b25e873 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:10.978403 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:11.026029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df401aef-9329-41bf-88a8-7a7f71696b34 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0dfcbad22348fdaea32d64557e68a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:11.026639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df401aef-9329-41bf-88a8-7a7f71696b34 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:11.026917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df401aef-9329-41bf-88a8-7a7f71696b34 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:11.027217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df401aef-9329-41bf-88a8-7a7f71696b34 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:11.029941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-df401aef-9329-41bf-88a8-7a7f71696b34 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:11.045716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 706/1394] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:16:11.111892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:11.146623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df401aef-9329-41bf-88a8-7a7f71696b34 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:11.148107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df401aef-9329-41bf-88a8-7a7f71696b34 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:11.167126 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 689/1395] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:11.486129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a011520-a4a9-422d-a7b8-c7748a31de3c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:11.487352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3a011520-a4a9-422d-a7b8-c7748a31de3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:11.652783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a011520-a4a9-422d-a7b8-c7748a31de3c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:11.782789 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:11.862831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc579a2-135f-4dfc-b893-851f29fde5b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:11.863455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc579a2-135f-4dfc-b893-851f29fde5b9 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:11.863733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc579a2-135f-4dfc-b893-851f29fde5b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:11.864003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc579a2-135f-4dfc-b893-851f29fde5b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:11.865744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6dc579a2-135f-4dfc-b893-851f29fde5b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:11.970101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc579a2-135f-4dfc-b893-851f29fde5b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:11.971727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc579a2-135f-4dfc-b893-851f29fde5b9 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:11.981876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fd9be-970d-4b6e-aae1-fe05c55d0bb9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:11.982270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fd9be-970d-4b6e-aae1-fe05c55d0bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:11.982579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fd9be-970d-4b6e-aae1-fe05c55d0bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:11.991310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 707/1396] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:12.057412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f26fd9be-970d-4b6e-aae1-fe05c55d0bb9 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:12.059005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f26fd9be-970d-4b6e-aae1-fe05c55d0bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:12.164564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 690/1397] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:16:12.214107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e45f83-23dc-4f8f-bc43-cee259355b50 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:12.214566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e45f83-23dc-4f8f-bc43-cee259355b50 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:12.214899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e45f83-23dc-4f8f-bc43-cee259355b50 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:12.217184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 708/1398] 10.222.0.75 () {54 vars in 943 bytes} [Fri Aug 19 15:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:16:12.224654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22db7f-706e-4a8d-ae73-c18fa9dbdc20 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:12.225097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22db7f-706e-4a8d-ae73-c18fa9dbdc20 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:12.225539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22db7f-706e-4a8d-ae73-c18fa9dbdc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:12.293603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:12.317357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2d22db7f-706e-4a8d-ae73-c18fa9dbdc20 None None] MFA Rules not processed for user `383ff491387642a3ab45ab692f94189c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:12.318579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2d22db7f-706e-4a8d-ae73-c18fa9dbdc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:12.440354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 691/1399] 10.222.0.75 () {58 vars in 1040 bytes} [Fri Aug 19 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:12.449817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e45f83-23dc-4f8f-bc43-cee259355b50 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:12.451383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d5e45f83-23dc-4f8f-bc43-cee259355b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:12.535487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:12.598470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e45f83-23dc-4f8f-bc43-cee259355b50 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:12.730738 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ee67-1901-415f-81ac-c6f31e55c6de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:12.731344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ee67-1901-415f-81ac-c6f31e55c6de service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:12.731815 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ee67-1901-415f-81ac-c6f31e55c6de service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:12.731976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ee67-1901-415f-81ac-c6f31e55c6de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:12.733734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8c4ee67-1901-415f-81ac-c6f31e55c6de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:12.804850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:12.828122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4ee67-1901-415f-81ac-c6f31e55c6de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:12.829919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4ee67-1901-415f-81ac-c6f31e55c6de service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:12.838558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 709/1400] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:16:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 15:16:14.505483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f775f8-6b31-4b8b-94c8-7d555c59bcc7 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:14.506027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f775f8-6b31-4b8b-94c8-7d555c59bcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:14.506027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f775f8-6b31-4b8b-94c8-7d555c59bcc7 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:14.507443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 692/1401] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:14.516250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6df94-e379-46ca-b833-0c6f017b2311 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:14.516816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6df94-e379-46ca-b833-0c6f017b2311 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:14.517121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6df94-e379-46ca-b833-0c6f017b2311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:14.603799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7ca6df94-e379-46ca-b833-0c6f017b2311 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:14.604986 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7ca6df94-e379-46ca-b833-0c6f017b2311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:14.736195 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 710/1402] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:15.032245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f775f8-6b31-4b8b-94c8-7d555c59bcc7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:15.034238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-63f775f8-6b31-4b8b-94c8-7d555c59bcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:15.156169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:15.218295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f775f8-6b31-4b8b-94c8-7d555c59bcc7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:15.377629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bed8b7-a360-43de-acd0-7fa7342b8219 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:15.378290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bed8b7-a360-43de-acd0-7fa7342b8219 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:15.378561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bed8b7-a360-43de-acd0-7fa7342b8219 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:15.378837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bed8b7-a360-43de-acd0-7fa7342b8219 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:15.380513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f4bed8b7-a360-43de-acd0-7fa7342b8219 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:15.465274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:15.490121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bed8b7-a360-43de-acd0-7fa7342b8219 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:15.491767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bed8b7-a360-43de-acd0-7fa7342b8219 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:15.514641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 693/1403] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:17.755638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19616f2e-e62d-49a1-a224-e3963e83ebad None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:17.755638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19616f2e-e62d-49a1-a224-e3963e83ebad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:17.755638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19616f2e-e62d-49a1-a224-e3963e83ebad None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:17.755638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 711/1404] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:17.760568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cb26e-f060-494f-a3cd-b3eb5f325f21 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:17.761365 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cb26e-f060-494f-a3cd-b3eb5f325f21 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:17.761365 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cb26e-f060-494f-a3cd-b3eb5f325f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:17.850829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f52cb26e-f060-494f-a3cd-b3eb5f325f21 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:17.853308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f52cb26e-f060-494f-a3cd-b3eb5f325f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:17.990640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 694/1405] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:16:18.291518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19616f2e-e62d-49a1-a224-e3963e83ebad None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:18.294088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-19616f2e-e62d-49a1-a224-e3963e83ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:18.397783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:18.516343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19616f2e-e62d-49a1-a224-e3963e83ebad None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:18.676689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02796927-7b59-4501-b659-12305e3c6398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:18.677304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796927-7b59-4501-b659-12305e3c6398 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:18.677581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796927-7b59-4501-b659-12305e3c6398 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:18.677857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02796927-7b59-4501-b659-12305e3c6398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:18.679522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-02796927-7b59-4501-b659-12305e3c6398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:18.706112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:18.798104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02796927-7b59-4501-b659-12305e3c6398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:18.799629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02796927-7b59-4501-b659-12305e3c6398 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:18.819755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 712/1406] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:20.928187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22580e3e-5922-4218-9b38-ee10b180338c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:20.928791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22580e3e-5922-4218-9b38-ee10b180338c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:20.928791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22580e3e-5922-4218-9b38-ee10b180338c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:20.930326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 695/1407] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:20.937019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a928ad4-da55-42c1-b049-19f6d6198074 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:20.937560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a928ad4-da55-42c1-b049-19f6d6198074 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:20.937806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a928ad4-da55-42c1-b049-19f6d6198074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:21.010563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0a928ad4-da55-42c1-b049-19f6d6198074 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:21.011927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a928ad4-da55-42c1-b049-19f6d6198074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:21.122665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:21.131800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 713/1408] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:21.428891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22580e3e-5922-4218-9b38-ee10b180338c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:21.430436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-22580e3e-5922-4218-9b38-ee10b180338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:21.492601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:21.627339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22580e3e-5922-4218-9b38-ee10b180338c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:21.762174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58f218-95dc-444e-be48-d97d30c0d128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:21.762852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58f218-95dc-444e-be48-d97d30c0d128 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:21.763132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58f218-95dc-444e-be48-d97d30c0d128 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:21.763412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58f218-95dc-444e-be48-d97d30c0d128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:21.765750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9a58f218-95dc-444e-be48-d97d30c0d128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:21.773403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:21.876383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58f218-95dc-444e-be48-d97d30c0d128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:21.878056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58f218-95dc-444e-be48-d97d30c0d128 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:21.897698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 696/1409] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:31.130327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:31.894477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=42 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:35.785566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88988167-44dc-461f-adae-fd40b53bfacc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:35.786389 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88988167-44dc-461f-adae-fd40b53bfacc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:35.786389 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88988167-44dc-461f-adae-fd40b53bfacc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:35.787465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 714/1410] 10.222.0.75 () {54 vars in 918 bytes} [Fri Aug 19 15:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:35.792665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8018516-f631-41df-b561-fcbd543e3370 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:35.793750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8018516-f631-41df-b561-fcbd543e3370 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:35.793750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8018516-f631-41df-b561-fcbd543e3370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:35.870426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f8018516-f631-41df-b561-fcbd543e3370 None None] MFA Rules not processed for user `bc2920aac4c046aca75473158ec204cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:35.871894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f8018516-f631-41df-b561-fcbd543e3370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:35.980964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 697/1411] 10.222.0.75 () {58 vars in 1015 bytes} [Fri Aug 19 15:16:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:35.995669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a57594-b623-427e-acbe-0b260f9aa7bb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:35.995952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a57594-b623-427e-acbe-0b260f9aa7bb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:35.996278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a57594-b623-427e-acbe-0b260f9aa7bb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:35.998338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 715/1412] 10.222.0.75 () {54 vars in 918 bytes} [Fri Aug 19 15:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:36.003161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e0b7-bfff-4e56-af2d-e77ce0286ba4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:36.003438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e0b7-bfff-4e56-af2d-e77ce0286ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:36.003748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e0b7-bfff-4e56-af2d-e77ce0286ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:36.077766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:36.080870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fa96e0b7-bfff-4e56-af2d-e77ce0286ba4 None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:36.082646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fa96e0b7-bfff-4e56-af2d-e77ce0286ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:36.202951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 698/1413] 10.222.0.75 () {58 vars in 1015 bytes} [Fri Aug 19 15:16:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:38.616345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827656b-7713-4743-a1e5-c15b323df123 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:38.616944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827656b-7713-4743-a1e5-c15b323df123 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:38.616944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827656b-7713-4743-a1e5-c15b323df123 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:38.618247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:38.624246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80615c08-bfcc-4ddd-aacd-b3d305a3afde None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:38.624726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80615c08-bfcc-4ddd-aacd-b3d305a3afde None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:38.624831 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80615c08-bfcc-4ddd-aacd-b3d305a3afde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:38.691974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-80615c08-bfcc-4ddd-aacd-b3d305a3afde None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:38.693202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-80615c08-bfcc-4ddd-aacd-b3d305a3afde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:38.748798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:38.791142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:39.080072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827656b-7713-4743-a1e5-c15b323df123 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:39.081750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3827656b-7713-4743-a1e5-c15b323df123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:39.226894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827656b-7713-4743-a1e5-c15b323df123 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:39.303867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:39.345223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792de88-cb8b-4807-99f9-cb1b3de6c45d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:39.345739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792de88-cb8b-4807-99f9-cb1b3de6c45d service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:39.346018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792de88-cb8b-4807-99f9-cb1b3de6c45d service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:39.346291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792de88-cb8b-4807-99f9-cb1b3de6c45d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:39.347886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b792de88-cb8b-4807-99f9-cb1b3de6c45d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:39.438856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792de88-cb8b-4807-99f9-cb1b3de6c45d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:39.440490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792de88-cb8b-4807-99f9-cb1b3de6c45d service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:39.459024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 717/1416] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:40.356802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d8f7e-b292-4988-b027-ae7af0263301 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:40.357366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d8f7e-b292-4988-b027-ae7af0263301 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:40.357366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d8f7e-b292-4988-b027-ae7af0263301 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:40.358682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:40.363659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44dbfb-2394-4235-9e27-94acb06e3e17 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:40.364098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44dbfb-2394-4235-9e27-94acb06e3e17 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:40.364239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44dbfb-2394-4235-9e27-94acb06e3e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:40.441529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6d44dbfb-2394-4235-9e27-94acb06e3e17 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:40.443199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6d44dbfb-2394-4235-9e27-94acb06e3e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:40.471810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:40.541540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:40.693158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d8f7e-b292-4988-b027-ae7af0263301 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:40.694677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-854d8f7e-b292-4988-b027-ae7af0263301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:40.838460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d8f7e-b292-4988-b027-ae7af0263301 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:40.879602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:40.953397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e386932-41dd-48a2-8cec-4d11459285fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:40.953997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e386932-41dd-48a2-8cec-4d11459285fd service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:40.954278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e386932-41dd-48a2-8cec-4d11459285fd service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:40.954551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e386932-41dd-48a2-8cec-4d11459285fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:40.956146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3e386932-41dd-48a2-8cec-4d11459285fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:41.053372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e386932-41dd-48a2-8cec-4d11459285fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:41.054978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e386932-41dd-48a2-8cec-4d11459285fd service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:41.073435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 701/1419] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:41.951637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64e07d-fce0-4c4c-8886-aeba4d55b137 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:41.952156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64e07d-fce0-4c4c-8886-aeba4d55b137 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:41.952156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64e07d-fce0-4c4c-8886-aeba4d55b137 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:41.953464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:41.958430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6ae99-e844-4b53-9b9d-74cf83a3b93b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:41.958811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6ae99-e844-4b53-9b9d-74cf83a3b93b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:41.958975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6ae99-e844-4b53-9b9d-74cf83a3b93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:42.017252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:42.040105 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c5e6ae99-e844-4b53-9b9d-74cf83a3b93b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:42.041346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c5e6ae99-e844-4b53-9b9d-74cf83a3b93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:42.138969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:42.299711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64e07d-fce0-4c4c-8886-aeba4d55b137 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:42.301266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb64e07d-fce0-4c4c-8886-aeba4d55b137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:42.446762 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64e07d-fce0-4c4c-8886-aeba4d55b137 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:42.458780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:42.561111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f1476-32e2-4072-a7a8-f72b9632d80a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:42.562262 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f1476-32e2-4072-a7a8-f72b9632d80a service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:42.562539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f1476-32e2-4072-a7a8-f72b9632d80a service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:42.562809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f1476-32e2-4072-a7a8-f72b9632d80a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:42.564430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-492f1476-32e2-4072-a7a8-f72b9632d80a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:42.653878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f1476-32e2-4072-a7a8-f72b9632d80a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:42.655500 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f1476-32e2-4072-a7a8-f72b9632d80a service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:42.673784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 720/1422] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:43.555136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d66433-037f-4499-93a6-78932d1ade32 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:43.555670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d66433-037f-4499-93a6-78932d1ade32 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:43.555670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d66433-037f-4499-93a6-78932d1ade32 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:43.557080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:43.562472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05191849-1dc4-48bf-b6ac-45b46255b9d2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:43.562885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05191849-1dc4-48bf-b6ac-45b46255b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:43.563024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05191849-1dc4-48bf-b6ac-45b46255b9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:43.583596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:43.645408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-05191849-1dc4-48bf-b6ac-45b46255b9d2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:43.646547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-05191849-1dc4-48bf-b6ac-45b46255b9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:43.743468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:43.896452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d66433-037f-4499-93a6-78932d1ade32 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:43.897914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e7d66433-037f-4499-93a6-78932d1ade32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:44.008996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:44.048614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d66433-037f-4499-93a6-78932d1ade32 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:44.165990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a677e9b-de6c-4cae-8ce5-a7729ddca5ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:44.166511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a677e9b-de6c-4cae-8ce5-a7729ddca5ca service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:44.166792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a677e9b-de6c-4cae-8ce5-a7729ddca5ca service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:44.167064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a677e9b-de6c-4cae-8ce5-a7729ddca5ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:44.168861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4a677e9b-de6c-4cae-8ce5-a7729ddca5ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:44.249583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:44.263862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a677e9b-de6c-4cae-8ce5-a7729ddca5ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:44.265437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a677e9b-de6c-4cae-8ce5-a7729ddca5ca service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:44.283874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 704/1425] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:45.153615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5ae46-d5c1-49e3-8325-105acf71c730 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:45.154143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5ae46-d5c1-49e3-8325-105acf71c730 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:45.154143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5ae46-d5c1-49e3-8325-105acf71c730 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:45.155562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:45.160995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d23cb2-6988-4569-8f76-74a18b184094 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:45.161391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d23cb2-6988-4569-8f76-74a18b184094 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:45.161548 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d23cb2-6988-4569-8f76-74a18b184094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:45.237950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c3d23cb2-6988-4569-8f76-74a18b184094 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:45.239171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c3d23cb2-6988-4569-8f76-74a18b184094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:45.336774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:45.489730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5ae46-d5c1-49e3-8325-105acf71c730 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:45.491238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-abd5ae46-d5c1-49e3-8325-105acf71c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:45.573955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:45.637733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5ae46-d5c1-49e3-8325-105acf71c730 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:45.760327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453421f-480c-470b-9ab4-325a3b7b8fed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:45.760958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453421f-480c-470b-9ab4-325a3b7b8fed service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:45.761240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453421f-480c-470b-9ab4-325a3b7b8fed service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:45.761514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453421f-480c-470b-9ab4-325a3b7b8fed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:45.763730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9453421f-480c-470b-9ab4-325a3b7b8fed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:45.817674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:45.859098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453421f-480c-470b-9ab4-325a3b7b8fed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:45.860773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453421f-480c-470b-9ab4-325a3b7b8fed service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:45.878954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 723/1428] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:46.683160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc41e76-bd95-465a-a4ec-41f5c32387bc None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:46.683635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc41e76-bd95-465a-a4ec-41f5c32387bc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:46.683757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc41e76-bd95-465a-a4ec-41f5c32387bc None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:46.685614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:46.690661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f835-17fa-4772-8164-02200b118952 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:46.691056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f835-17fa-4772-8164-02200b118952 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:46.691212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8f835-17fa-4772-8164-02200b118952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:46.766516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-01c8f835-17fa-4772-8164-02200b118952 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:46.767769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-01c8f835-17fa-4772-8164-02200b118952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:46.864086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:47.019378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc41e76-bd95-465a-a4ec-41f5c32387bc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:47.020936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-afc41e76-bd95-465a-a4ec-41f5c32387bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:47.070392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:47.169444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc41e76-bd95-465a-a4ec-41f5c32387bc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:47.281181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a31e5-039e-466a-b5fa-6af92edf145a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:47.281788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a31e5-039e-466a-b5fa-6af92edf145a service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:47.282071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a31e5-039e-466a-b5fa-6af92edf145a service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:47.282343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a31e5-039e-466a-b5fa-6af92edf145a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:47.283950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9a6a31e5-039e-466a-b5fa-6af92edf145a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:47.291081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:47.378212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a31e5-039e-466a-b5fa-6af92edf145a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:47.379843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a31e5-039e-466a-b5fa-6af92edf145a service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:47.397677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 707/1431] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:48.216824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d138e2-7edd-4268-9c7d-a105231c6f8d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:48.217496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d138e2-7edd-4268-9c7d-a105231c6f8d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:48.217496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d138e2-7edd-4268-9c7d-a105231c6f8d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:48.218698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:48.223686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a25188-6201-46b8-84dd-c1f2f66f9a25 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:48.224131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a25188-6201-46b8-84dd-c1f2f66f9a25 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:48.224245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a25188-6201-46b8-84dd-c1f2f66f9a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:48.302738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-29a25188-6201-46b8-84dd-c1f2f66f9a25 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:48.303937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-29a25188-6201-46b8-84dd-c1f2f66f9a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:48.372194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:48.403918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:16:48.560003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d138e2-7edd-4268-9c7d-a105231c6f8d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:48.561585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-41d138e2-7edd-4268-9c7d-a105231c6f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:48.580276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:48.711480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d138e2-7edd-4268-9c7d-a105231c6f8d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:48.799040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:48.823950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20d493-3c7c-46af-9818-f06e7df30a4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:48.825114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20d493-3c7c-46af-9818-f06e7df30a4b service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:48.825390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20d493-3c7c-46af-9818-f06e7df30a4b service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:48.825740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20d493-3c7c-46af-9818-f06e7df30a4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:48.827329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f20d493-3c7c-46af-9818-f06e7df30a4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:48.915312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20d493-3c7c-46af-9818-f06e7df30a4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:48.916953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20d493-3c7c-46af-9818-f06e7df30a4b service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:48.935122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 726/1434] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:49.748994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e5372-dd8d-42e4-b6d4-d52cf8125dd1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:49.749554 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e5372-dd8d-42e4-b6d4-d52cf8125dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:49.749554 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e5372-dd8d-42e4-b6d4-d52cf8125dd1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:49.750929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:49.756155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b9d5c-70c6-4d8d-970e-58123dbfdb94 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:49.756617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b9d5c-70c6-4d8d-970e-58123dbfdb94 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:49.756743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b9d5c-70c6-4d8d-970e-58123dbfdb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:49.837791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-740b9d5c-70c6-4d8d-970e-58123dbfdb94 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:49.838996 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-740b9d5c-70c6-4d8d-970e-58123dbfdb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:49.881355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:49.936215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:50.088947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e5372-dd8d-42e4-b6d4-d52cf8125dd1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:50.090481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9b8e5372-dd8d-42e4-b6d4-d52cf8125dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:50.235157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e5372-dd8d-42e4-b6d4-d52cf8125dd1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:50.289558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:50.351277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72a556-5dd2-40e2-b99e-9d4fe7e891e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:50.351882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72a556-5dd2-40e2-b99e-9d4fe7e891e7 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:50.352166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72a556-5dd2-40e2-b99e-9d4fe7e891e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:50.352467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72a556-5dd2-40e2-b99e-9d4fe7e891e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:50.354077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cd72a556-5dd2-40e2-b99e-9d4fe7e891e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:50.443203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72a556-5dd2-40e2-b99e-9d4fe7e891e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:50.444860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72a556-5dd2-40e2-b99e-9d4fe7e891e7 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:50.463059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 710/1437] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:51.277874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc66cab-b4db-4113-aca5-c20eb65c3ef3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:51.278479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc66cab-b4db-4113-aca5-c20eb65c3ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:51.278479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc66cab-b4db-4113-aca5-c20eb65c3ef3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:51.279759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:16:51.285368 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e537e59-4d72-4fb9-9e95-2762d5fa7a1e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:51.285862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e537e59-4d72-4fb9-9e95-2762d5fa7a1e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:51.286034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e537e59-4d72-4fb9-9e95-2762d5fa7a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:51.364624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3e537e59-4d72-4fb9-9e95-2762d5fa7a1e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:51.365839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3e537e59-4d72-4fb9-9e95-2762d5fa7a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:51.367898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:51.464835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:51.616923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc66cab-b4db-4113-aca5-c20eb65c3ef3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:51.618384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5dc66cab-b4db-4113-aca5-c20eb65c3ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:51.765217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc66cab-b4db-4113-aca5-c20eb65c3ef3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:51.794390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:51.880799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15a97a-5352-4b28-8a93-94e8f341a851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:51.881379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a97a-5352-4b28-8a93-94e8f341a851 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:51.881663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a97a-5352-4b28-8a93-94e8f341a851 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:51.881935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a97a-5352-4b28-8a93-94e8f341a851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:51.883560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0d15a97a-5352-4b28-8a93-94e8f341a851 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:51.974830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15a97a-5352-4b28-8a93-94e8f341a851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:51.976532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15a97a-5352-4b28-8a93-94e8f341a851 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:51.994492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 729/1440] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:53.857713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3273-e56b-4e0a-b7dd-328592216375 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:53.858364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3273-e56b-4e0a-b7dd-328592216375 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:53.858364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3273-e56b-4e0a-b7dd-328592216375 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:53.859729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:16:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:16:53.865698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c053a-1078-43b3-b7ff-9098624bd11d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:53.866743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c053a-1078-43b3-b7ff-9098624bd11d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:53.866743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c053a-1078-43b3-b7ff-9098624bd11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:53.907995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:53.934104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7d6c053a-1078-43b3-b7ff-9098624bd11d None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:53.935967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d6c053a-1078-43b3-b7ff-9098624bd11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:54.034072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:54.322636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3273-e56b-4e0a-b7dd-328592216375 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:54.324201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fc2c3273-e56b-4e0a-b7dd-328592216375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:54.426799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3273-e56b-4e0a-b7dd-328592216375 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:54.460601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:54.503950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c1b27-57bc-47e2-97dc-83b1a8863b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:54.504596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c1b27-57bc-47e2-97dc-83b1a8863b56 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:54.504878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c1b27-57bc-47e2-97dc-83b1a8863b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:54.505148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c1b27-57bc-47e2-97dc-83b1a8863b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:54.506769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d22c1b27-57bc-47e2-97dc-83b1a8863b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:54.616773 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c1b27-57bc-47e2-97dc-83b1a8863b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:54.618162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c1b27-57bc-47e2-97dc-83b1a8863b56 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:54.636236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 713/1443] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:56.722814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cb0ea-138e-454b-b69a-1cf5bbfff31f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:56.723363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cb0ea-138e-454b-b69a-1cf5bbfff31f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:56.723363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cb0ea-138e-454b-b69a-1cf5bbfff31f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:56.728510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:56.730241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b245490-e810-49fc-8246-29cf23705e02 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:56.730667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b245490-e810-49fc-8246-29cf23705e02 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:56.730841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b245490-e810-49fc-8246-29cf23705e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:56.802670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9b245490-e810-49fc-8246-29cf23705e02 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:56.803870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9b245490-e810-49fc-8246-29cf23705e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:56.817004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:56.907122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:16:57.193376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cb0ea-138e-454b-b69a-1cf5bbfff31f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:16:57.194900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-509cb0ea-138e-454b-b69a-1cf5bbfff31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:57.299143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cb0ea-138e-454b-b69a-1cf5bbfff31f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:16:57.311897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:57.378152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2bb74-9017-48fc-b4cd-37b518002488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:16:57.378781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2bb74-9017-48fc-b4cd-37b518002488 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:57.379049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2bb74-9017-48fc-b4cd-37b518002488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:57.379321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2bb74-9017-48fc-b4cd-37b518002488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:57.380975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-47b2bb74-9017-48fc-b4cd-37b518002488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:57.494202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2bb74-9017-48fc-b4cd-37b518002488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:16:57.495651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2bb74-9017-48fc-b4cd-37b518002488 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:16:57.517581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 732/1446] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:16:59.819881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d07cb-42b2-4cb3-a7d8-0398224bddb6 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:59.820764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d07cb-42b2-4cb3-a7d8-0398224bddb6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:59.820764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d07cb-42b2-4cb3-a7d8-0398224bddb6 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:59.821860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:16:59.827894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97a4e-3284-40aa-ab84-b348ccebd9ec None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:16:59.828340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97a4e-3284-40aa-ab84-b348ccebd9ec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:16:59.828556 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c97a4e-3284-40aa-ab84-b348ccebd9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:16:59.875545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:16:59.894543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-58c97a4e-3284-40aa-ab84-b348ccebd9ec None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:16:59.895737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-58c97a4e-3284-40aa-ab84-b348ccebd9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:16:59.997784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:00.283567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d07cb-42b2-4cb3-a7d8-0398224bddb6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:00.285383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-405d07cb-42b2-4cb3-a7d8-0398224bddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:00.386772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d07cb-42b2-4cb3-a7d8-0398224bddb6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:00.427698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:00.462367 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec853801-8918-4204-bf09-293615d553b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:00.462968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec853801-8918-4204-bf09-293615d553b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:00.463250 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec853801-8918-4204-bf09-293615d553b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:00.463524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec853801-8918-4204-bf09-293615d553b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:00.465157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ec853801-8918-4204-bf09-293615d553b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:00.573437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec853801-8918-4204-bf09-293615d553b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:00.574800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec853801-8918-4204-bf09-293615d553b9 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:00.593931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 716/1449] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:02.663063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887e89d-b029-4995-841b-630f1e02638e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:02.663613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887e89d-b029-4995-841b-630f1e02638e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:02.663613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887e89d-b029-4995-841b-630f1e02638e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:02.664919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:17:02.670705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6f2d-ab18-4585-a7e5-4651141b0428 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:02.671160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6f2d-ab18-4585-a7e5-4651141b0428 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:02.671266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e6f2d-ab18-4585-a7e5-4651141b0428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:02.737574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f54e6f2d-ab18-4585-a7e5-4651141b0428 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:02.738755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f54e6f2d-ab18-4585-a7e5-4651141b0428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:02.759988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:02.836212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:03.131102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887e89d-b029-4995-841b-630f1e02638e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:03.132696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d887e89d-b029-4995-841b-630f1e02638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:03.244612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887e89d-b029-4995-841b-630f1e02638e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:03.265702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:03.321432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d74e7c-01ec-4a17-b757-d4585d1b5994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:03.322040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d74e7c-01ec-4a17-b757-d4585d1b5994 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:03.322384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d74e7c-01ec-4a17-b757-d4585d1b5994 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:03.322592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d74e7c-01ec-4a17-b757-d4585d1b5994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:03.324200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-05d74e7c-01ec-4a17-b757-d4585d1b5994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:03.445598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d74e7c-01ec-4a17-b757-d4585d1b5994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:03.446986 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d74e7c-01ec-4a17-b757-d4585d1b5994 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:03.465849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 735/1452] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:05.617508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb60f8-8423-439a-a72b-a55152e5116d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:05.618096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb60f8-8423-439a-a72b-a55152e5116d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:05.618096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb60f8-8423-439a-a72b-a55152e5116d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:05.619399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:05.625635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9dec8-77da-4189-92ad-9dd8ee154049 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:05.626127 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9dec8-77da-4189-92ad-9dd8ee154049 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:05.626243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9dec8-77da-4189-92ad-9dd8ee154049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:05.693469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:05.696217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-40b9dec8-77da-4189-92ad-9dd8ee154049 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:05.697963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-40b9dec8-77da-4189-92ad-9dd8ee154049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:05.795814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:06.084650 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb60f8-8423-439a-a72b-a55152e5116d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:06.086189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e2eb60f8-8423-439a-a72b-a55152e5116d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:06.191828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb60f8-8423-439a-a72b-a55152e5116d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:06.240070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:06.265522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805df1c-928f-4769-bd2f-ded133d11bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:06.266121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805df1c-928f-4769-bd2f-ded133d11bcc service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:06.266428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805df1c-928f-4769-bd2f-ded133d11bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:06.266700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805df1c-928f-4769-bd2f-ded133d11bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:06.268330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3805df1c-928f-4769-bd2f-ded133d11bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:06.379457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805df1c-928f-4769-bd2f-ded133d11bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:06.380852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805df1c-928f-4769-bd2f-ded133d11bcc service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:06.398820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 719/1455] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:08.481894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8a59d-66f6-4f04-8f60-3b20b0f22ffc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:08.482458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8a59d-66f6-4f04-8f60-3b20b0f22ffc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:08.482458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8a59d-66f6-4f04-8f60-3b20b0f22ffc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:08.483906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:08.490068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf312273-cd86-4ac6-bf80-f2e6c66ad230 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:08.490464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf312273-cd86-4ac6-bf80-f2e6c66ad230 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:08.490608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf312273-cd86-4ac6-bf80-f2e6c66ad230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:08.557838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bf312273-cd86-4ac6-bf80-f2e6c66ad230 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:08.558959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bf312273-cd86-4ac6-bf80-f2e6c66ad230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:08.588430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:08.657459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:08.948126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8a59d-66f6-4f04-8f60-3b20b0f22ffc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:08.949810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-31e8a59d-66f6-4f04-8f60-3b20b0f22ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:09.055832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8a59d-66f6-4f04-8f60-3b20b0f22ffc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:09.084365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:09.129725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612d8cf-e58e-4a73-8453-f92027dab2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:09.130327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612d8cf-e58e-4a73-8453-f92027dab2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:09.130603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612d8cf-e58e-4a73-8453-f92027dab2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:09.130873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612d8cf-e58e-4a73-8453-f92027dab2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:09.132490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8612d8cf-e58e-4a73-8453-f92027dab2e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:09.245850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612d8cf-e58e-4a73-8453-f92027dab2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:09.247807 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612d8cf-e58e-4a73-8453-f92027dab2e6 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:09.266329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 738/1458] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:11.222609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a557570-87f1-447c-8f89-d2322dfdf52a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:11.223184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a557570-87f1-447c-8f89-d2322dfdf52a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:11.223184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a557570-87f1-447c-8f89-d2322dfdf52a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:11.224545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:11.229977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d7e0-35cf-4bb7-9a25-4f6ea2c5aadf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:11.233600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d7e0-35cf-4bb7-9a25-4f6ea2c5aadf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:11.233600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d7e0-35cf-4bb7-9a25-4f6ea2c5aadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:11.297615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9e95d7e0-35cf-4bb7-9a25-4f6ea2c5aadf None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:11.298829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e95d7e0-35cf-4bb7-9a25-4f6ea2c5aadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:11.307047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:11.397445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:11.694423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a557570-87f1-447c-8f89-d2322dfdf52a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:11.695980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6a557570-87f1-447c-8f89-d2322dfdf52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:11.800156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a557570-87f1-447c-8f89-d2322dfdf52a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:11.854665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:11.875455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0f7b2-4150-4dbe-85f1-3618d5bccc0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:11.876056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0f7b2-4150-4dbe-85f1-3618d5bccc0c service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:11.876336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0f7b2-4150-4dbe-85f1-3618d5bccc0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:11.876637 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0f7b2-4150-4dbe-85f1-3618d5bccc0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:11.878244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-83c0f7b2-4150-4dbe-85f1-3618d5bccc0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:11.986025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0f7b2-4150-4dbe-85f1-3618d5bccc0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:11.987394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0f7b2-4150-4dbe-85f1-3618d5bccc0c service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:12.006209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 722/1461] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:13.997948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d5df7-a4be-4665-ab72-5111578be526 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:13.998479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d5df7-a4be-4665-ab72-5111578be526 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:13.998479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d5df7-a4be-4665-ab72-5111578be526 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:13.999725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:14.005205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fffbea-66aa-44aa-b56b-f1468c853b1c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:14.005654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fffbea-66aa-44aa-b56b-f1468c853b1c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:14.005795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fffbea-66aa-44aa-b56b-f1468c853b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:14.071930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f5fffbea-66aa-44aa-b56b-f1468c853b1c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:14.073233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f5fffbea-66aa-44aa-b56b-f1468c853b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:14.111270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:14.171252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:14.456910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d5df7-a4be-4665-ab72-5111578be526 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:14.458485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e15d5df7-a4be-4665-ab72-5111578be526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:14.561654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d5df7-a4be-4665-ab72-5111578be526 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:14.599789 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:14.635274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30adaca-a73c-4676-9587-58ad22f311e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:14.635905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30adaca-a73c-4676-9587-58ad22f311e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:14.636185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30adaca-a73c-4676-9587-58ad22f311e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:14.636492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30adaca-a73c-4676-9587-58ad22f311e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:14.638154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f30adaca-a73c-4676-9587-58ad22f311e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:14.750616 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30adaca-a73c-4676-9587-58ad22f311e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:14.752070 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30adaca-a73c-4676-9587-58ad22f311e2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:14.770605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 741/1464] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:16.705547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c3957-f0b0-4769-b4f2-60ce2f8b5f16 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:16.706026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c3957-f0b0-4769-b4f2-60ce2f8b5f16 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:16.706143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c3957-f0b0-4769-b4f2-60ce2f8b5f16 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:16.707430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:16.712725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0eebf0-10dd-4e1c-9346-49296bc5175f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:16.713153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0eebf0-10dd-4e1c-9346-49296bc5175f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:16.713250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0eebf0-10dd-4e1c-9346-49296bc5175f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:16.780043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5a0eebf0-10dd-4e1c-9346-49296bc5175f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:16.781417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5a0eebf0-10dd-4e1c-9346-49296bc5175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:16.800236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:16.880164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:17.172248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c3957-f0b0-4769-b4f2-60ce2f8b5f16 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:17.174026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-622c3957-f0b0-4769-b4f2-60ce2f8b5f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:17.279264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c3957-f0b0-4769-b4f2-60ce2f8b5f16 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:17.347144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:17.354387 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664474dc-d48a-40b7-b9f0-d5fdbc581fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:17.354984 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664474dc-d48a-40b7-b9f0-d5fdbc581fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:17.355259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664474dc-d48a-40b7-b9f0-d5fdbc581fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:17.355527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664474dc-d48a-40b7-b9f0-d5fdbc581fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:17.357300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-664474dc-d48a-40b7-b9f0-d5fdbc581fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:17.474317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664474dc-d48a-40b7-b9f0-d5fdbc581fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:17.475792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664474dc-d48a-40b7-b9f0-d5fdbc581fc3 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:17.494884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 725/1467] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:19.426089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed9e7-cbfc-4e83-ba39-243c4528bf8f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:19.426758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed9e7-cbfc-4e83-ba39-243c4528bf8f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:19.426758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ed9e7-cbfc-4e83-ba39-243c4528bf8f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:19.428014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:19.433342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1b521-2ac9-4644-9657-1eb55402659e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:19.433784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1b521-2ac9-4644-9657-1eb55402659e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:19.433919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1b521-2ac9-4644-9657-1eb55402659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:19.501076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f2a1b521-2ac9-4644-9657-1eb55402659e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:19.502318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f2a1b521-2ac9-4644-9657-1eb55402659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:19.556771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:19.615898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:19.900326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ed9e7-cbfc-4e83-ba39-243c4528bf8f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:19.901889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c41ed9e7-cbfc-4e83-ba39-243c4528bf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:20.007732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ed9e7-cbfc-4e83-ba39-243c4528bf8f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:20.056524 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:20.082483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd4a7b-be5b-463c-b30f-2c11e3025943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:20.083088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd4a7b-be5b-463c-b30f-2c11e3025943 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:20.083366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd4a7b-be5b-463c-b30f-2c11e3025943 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:20.083639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd4a7b-be5b-463c-b30f-2c11e3025943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:20.085257 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-55bd4a7b-be5b-463c-b30f-2c11e3025943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:20.197252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd4a7b-be5b-463c-b30f-2c11e3025943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:20.198615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd4a7b-be5b-463c-b30f-2c11e3025943 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:20.216354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 744/1470] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:29.613948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=22 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:30.214018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=66 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:40.040896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd6bbd-1b2f-4065-ae0e-3d62fa9b2edf None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:40.041463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd6bbd-1b2f-4065-ae0e-3d62fa9b2edf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:40.041463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd6bbd-1b2f-4065-ae0e-3d62fa9b2edf None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:40.042822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 727/1471] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:40.048699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ab2e4-30af-4601-8782-d37be9a5dd74 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:40.049109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ab2e4-30af-4601-8782-d37be9a5dd74 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:40.049285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ab2e4-30af-4601-8782-d37be9a5dd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:40.114950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7f9ab2e4-30af-4601-8782-d37be9a5dd74 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:40.116148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7f9ab2e4-30af-4601-8782-d37be9a5dd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:40.212434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 745/1472] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:40.510288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe50e6-6267-4825-9fbd-e365e799d0e0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:40.510574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe50e6-6267-4825-9fbd-e365e799d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:40.510851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe50e6-6267-4825-9fbd-e365e799d0e0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:40.512235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 728/1473] 10.222.0.75 () {54 vars in 967 bytes} [Fri Aug 19 15:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:40.517743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1cf7c-5932-468b-9b3d-072ad2b3b8b9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:40.518031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1cf7c-5932-468b-9b3d-072ad2b3b8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:40.518309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1cf7c-5932-468b-9b3d-072ad2b3b8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:40.594537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-45e1cf7c-5932-468b-9b3d-072ad2b3b8b9 None None] MFA Rules not processed for user `de0dfcbad22348fdaea32d64557e68a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:40.595645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-45e1cf7c-5932-468b-9b3d-072ad2b3b8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:40.616275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:40.672971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 746/1474] 10.222.0.75 () {58 vars in 1064 bytes} [Fri Aug 19 15:17:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:40.680222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe50e6-6267-4825-9fbd-e365e799d0e0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:40.681762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-42fe50e6-6267-4825-9fbd-e365e799d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:40.783818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe50e6-6267-4825-9fbd-e365e799d0e0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:40.858237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8bf17-c902-478a-b606-907399bf908c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0dfcbad22348fdaea32d64557e68a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:40.858772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8bf17-c902-478a-b606-907399bf908c service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:40.859054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8bf17-c902-478a-b606-907399bf908c service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:40.859328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8bf17-c902-478a-b606-907399bf908c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:40.860949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6cf8bf17-c902-478a-b606-907399bf908c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:40.914997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:40.970373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8bf17-c902-478a-b606-907399bf908c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:40.971822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8bf17-c902-478a-b606-907399bf908c service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:40.992012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 729/1475] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:42.984430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39947f-94d1-40fb-9c9e-fd8982a12e9c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:42.984963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39947f-94d1-40fb-9c9e-fd8982a12e9c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:42.984963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39947f-94d1-40fb-9c9e-fd8982a12e9c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:42.986290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 747/1476] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:42.991470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bf68a-6a21-4d58-afab-e38060c9cffd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:42.991939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bf68a-6a21-4d58-afab-e38060c9cffd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:42.992048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bf68a-6a21-4d58-afab-e38060c9cffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:43.057253 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-046bf68a-6a21-4d58-afab-e38060c9cffd None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:43.058454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-046bf68a-6a21-4d58-afab-e38060c9cffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:43.152616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 730/1477] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:43.423587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39947f-94d1-40fb-9c9e-fd8982a12e9c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:43.425147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fe39947f-94d1-40fb-9c9e-fd8982a12e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:43.551597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39947f-94d1-40fb-9c9e-fd8982a12e9c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:43.602555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:43.652202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef78c28-e695-4bf3-85c2-9436d505fa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:43.652833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef78c28-e695-4bf3-85c2-9436d505fa14 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:43.653303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef78c28-e695-4bf3-85c2-9436d505fa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:43.653578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef78c28-e695-4bf3-85c2-9436d505fa14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:43.654210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef78c28-e695-4bf3-85c2-9436d505fa14 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:43.655976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-6ef78c28-e695-4bf3-85c2-9436d505fa14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:17:43.655976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 19 15:17:43.655976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: Aug 19 15:17:43.657158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef78c28-e695-4bf3-85c2-9436d505fa14 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:43.676236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 748/1478] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Aug 19 15:17:43 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:17:45.617962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526f5eb-c5c8-4916-bd4f-a577dda5dce0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:45.618474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526f5eb-c5c8-4916-bd4f-a577dda5dce0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:45.618474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526f5eb-c5c8-4916-bd4f-a577dda5dce0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:45.620205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 731/1479] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:45.625801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ef65b-8384-4161-88d8-f46f222235ca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:45.626234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ef65b-8384-4161-88d8-f46f222235ca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:45.626336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ef65b-8384-4161-88d8-f46f222235ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:45.692800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-945ef65b-8384-4161-88d8-f46f222235ca None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:45.693997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-945ef65b-8384-4161-88d8-f46f222235ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:45.794644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 749/1480] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:46.098339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526f5eb-c5c8-4916-bd4f-a577dda5dce0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:46.100521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f526f5eb-c5c8-4916-bd4f-a577dda5dce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:46.110809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:46.252824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526f5eb-c5c8-4916-bd4f-a577dda5dce0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:46.334960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:46.367946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9d961-562e-483b-a8b7-75d58fbc3273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:46.368585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9d961-562e-483b-a8b7-75d58fbc3273 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:46.368869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9d961-562e-483b-a8b7-75d58fbc3273 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:46.369145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9d961-562e-483b-a8b7-75d58fbc3273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:46.370743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c2b9d961-562e-483b-a8b7-75d58fbc3273 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:46.460916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9d961-562e-483b-a8b7-75d58fbc3273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:46.462534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9d961-562e-483b-a8b7-75d58fbc3273 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:46.480511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 732/1481] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:48.608634 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67a355-3470-4802-a4b7-cbb9e49de654 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:48.609146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67a355-3470-4802-a4b7-cbb9e49de654 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:48.609146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67a355-3470-4802-a4b7-cbb9e49de654 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:48.610441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 750/1482] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:48.615714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71fe14-5263-4fd9-82af-78ed879f2f6f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:48.616117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71fe14-5263-4fd9-82af-78ed879f2f6f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:48.616268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71fe14-5263-4fd9-82af-78ed879f2f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:48.802906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ad71fe14-5263-4fd9-82af-78ed879f2f6f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:48.804122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ad71fe14-5263-4fd9-82af-78ed879f2f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:48.853671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:48.904738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 733/1483] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:49.195970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67a355-3470-4802-a4b7-cbb9e49de654 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:49.197586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9b67a355-3470-4802-a4b7-cbb9e49de654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:49.223274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:49.305241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67a355-3470-4802-a4b7-cbb9e49de654 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:49.379602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c990035-3a12-4eab-9900-5fa1acad25bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:49.380208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c990035-3a12-4eab-9900-5fa1acad25bd service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:49.380521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c990035-3a12-4eab-9900-5fa1acad25bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:49.380795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c990035-3a12-4eab-9900-5fa1acad25bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:49.382417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1c990035-3a12-4eab-9900-5fa1acad25bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:49.459506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:49.495134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c990035-3a12-4eab-9900-5fa1acad25bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:49.496679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c990035-3a12-4eab-9900-5fa1acad25bd service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:49.514739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 751/1484] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:51.498021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd864358-2de9-46e1-a52b-93aaad699e29 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:51.498576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd864358-2de9-46e1-a52b-93aaad699e29 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:51.498576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd864358-2de9-46e1-a52b-93aaad699e29 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:51.499789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 734/1485] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:51.505033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4100498-09bd-4090-9b01-ad6668dceab5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:51.505482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4100498-09bd-4090-9b01-ad6668dceab5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:51.505597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4100498-09bd-4090-9b01-ad6668dceab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:51.572064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c4100498-09bd-4090-9b01-ad6668dceab5 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:51.573309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c4100498-09bd-4090-9b01-ad6668dceab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:51.670950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 752/1486] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:51.959554 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd864358-2de9-46e1-a52b-93aaad699e29 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:51.961098 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bd864358-2de9-46e1-a52b-93aaad699e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:52.064371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd864358-2de9-46e1-a52b-93aaad699e29 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:52.136747 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:52.139258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71049b49-c06a-4fee-abda-ee7c74dcc299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:52.140128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71049b49-c06a-4fee-abda-ee7c74dcc299 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:52.140698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71049b49-c06a-4fee-abda-ee7c74dcc299 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:52.141145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71049b49-c06a-4fee-abda-ee7c74dcc299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:52.143003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-71049b49-c06a-4fee-abda-ee7c74dcc299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:52.251294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71049b49-c06a-4fee-abda-ee7c74dcc299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:52.252673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71049b49-c06a-4fee-abda-ee7c74dcc299 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:52.272034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 735/1487] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:54.304462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2ec28-3039-45c6-b635-07abd417fa82 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:54.304876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2ec28-3039-45c6-b635-07abd417fa82 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:54.304988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2ec28-3039-45c6-b635-07abd417fa82 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:54.306456 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 753/1488] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:54.312664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d945eb-7026-4c40-a24d-4ab0be500d44 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:54.313125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d945eb-7026-4c40-a24d-4ab0be500d44 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:54.313236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d945eb-7026-4c40-a24d-4ab0be500d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:54.379947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e2d945eb-7026-4c40-a24d-4ab0be500d44 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:54.381184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e2d945eb-7026-4c40-a24d-4ab0be500d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:54.435389 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:54.479854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 736/1489] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:54.783273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2ec28-3039-45c6-b635-07abd417fa82 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:54.784885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-28e2ec28-3039-45c6-b635-07abd417fa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:54.821323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:54.897749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2ec28-3039-45c6-b635-07abd417fa82 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:54.972124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80616872-f8bd-4bf4-8456-0524ae225fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:54.972669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80616872-f8bd-4bf4-8456-0524ae225fef service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:54.972951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80616872-f8bd-4bf4-8456-0524ae225fef service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:54.973223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80616872-f8bd-4bf4-8456-0524ae225fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:54.974843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-80616872-f8bd-4bf4-8456-0524ae225fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:55.057220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:55.087008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80616872-f8bd-4bf4-8456-0524ae225fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:55.088506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80616872-f8bd-4bf4-8456-0524ae225fef service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:55.107350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 754/1490] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:17:57.235396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7fecc-cbe0-4b02-88e8-bed53637eea9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:57.235938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7fecc-cbe0-4b02-88e8-bed53637eea9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:57.235938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7fecc-cbe0-4b02-88e8-bed53637eea9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:57.237345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 737/1491] 10.222.0.75 () {54 vars in 927 bytes} [Fri Aug 19 15:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:17:57.242754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f79ae-c493-4630-8289-3c1231b92c75 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:57.243165 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f79ae-c493-4630-8289-3c1231b92c75 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:57.243314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f79ae-c493-4630-8289-3c1231b92c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:57.311950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-090f79ae-c493-4630-8289-3c1231b92c75 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:17:57.313220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-090f79ae-c493-4630-8289-3c1231b92c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:57.412360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 755/1492] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Aug 19 15:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:17:57.714519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7fecc-cbe0-4b02-88e8-bed53637eea9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:17:57.716593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-63c7fecc-cbe0-4b02-88e8-bed53637eea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:57.819946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7fecc-cbe0-4b02-88e8-bed53637eea9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:17:57.900028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab4bb9-0427-47ae-a58c-8e0a486cb580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:17:57.900666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab4bb9-0427-47ae-a58c-8e0a486cb580 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:17:57.900944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab4bb9-0427-47ae-a58c-8e0a486cb580 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:17:57.901216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab4bb9-0427-47ae-a58c-8e0a486cb580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:17:57.903530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f2ab4bb9-0427-47ae-a58c-8e0a486cb580 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:17:57.913107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:17:58.013167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab4bb9-0427-47ae-a58c-8e0a486cb580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:17:58.014616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab4bb9-0427-47ae-a58c-8e0a486cb580 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:17:58.033147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 738/1493] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:18:07.410344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:18:08.030564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=50 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:24.122618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3774f3-c1d2-4baf-80cf-00c54ae133ab None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:24.123303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3774f3-c1d2-4baf-80cf-00c54ae133ab None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:24.123303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3774f3-c1d2-4baf-80cf-00c54ae133ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:24.205979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cd3774f3-c1d2-4baf-80cf-00c54ae133ab None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:19:24.207200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cd3774f3-c1d2-4baf-80cf-00c54ae133ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:24.315535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 756/1494] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:19:24.323600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82b0dc-74f5-4cd0-a77c-344f46998fef None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:24.324153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82b0dc-74f5-4cd0-a77c-344f46998fef None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:24.324153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82b0dc-74f5-4cd0-a77c-344f46998fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:24.404824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4d82b0dc-74f5-4cd0-a77c-344f46998fef None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:19:24.405928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4d82b0dc-74f5-4cd0-a77c-344f46998fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:24.506454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 739/1495] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:19:24.511185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3774f3-c1d2-4baf-80cf-00c54ae133ab None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:24.512454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cd3774f3-c1d2-4baf-80cf-00c54ae133ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:24.524260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3774f3-c1d2-4baf-80cf-00c54ae133ab None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:24.589849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:24.627572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:24.627572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:24.627572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:24.627572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:24.628110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:19:24.628110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:24.628283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:19:24.628283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 19 15:19:24.628283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: Aug 19 15:19:24.629227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:24.640733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 757/1496] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:19:24.651205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82b0dc-74f5-4cd0-a77c-344f46998fef None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:24.661932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82b0dc-74f5-4cd0-a77c-344f46998fef None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:24.725658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:24.778150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:24.778756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:24.779031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:24.779297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:24.779933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:24.781718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:19:24.781718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Aug 19 15:19:24.781718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: Aug 19 15:19:24.782914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:24.840856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 740/1497] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:19:24 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:19:24.845154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:24.846683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:24.881260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6474c-5b64-4bd9-8f01-40b491cccd56 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:24.987048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:24.987575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:24.987856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:24.988131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:24.989531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:24.990572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:19:24.990572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 19 15:19:24.990572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: Aug 19 15:19:24.994058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:25.045059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 758/1498] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:19:24 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:19:25.049343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:25.050865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:25.118967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4108b66-0f31-4cba-a60b-02a968c87301 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:25.208111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:25.221927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:25.222588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:25.222848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:25.223118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:25.223900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:25.225132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:19:25.225132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Aug 19 15:19:25.225132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: Aug 19 15:19:25.226036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:25.238855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 741/1499] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:19:25.243102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:25.244686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:25.257183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f68c1-e223-4687-bc2c-a64c78d831ab admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:25.291394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:25.368743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17afffe-480e-425d-9f3d-b4d38de09bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:25.369356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17afffe-480e-425d-9f3d-b4d38de09bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:25.369646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17afffe-480e-425d-9f3d-b4d38de09bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:25.369902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17afffe-480e-425d-9f3d-b4d38de09bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:25.371064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17afffe-480e-425d-9f3d-b4d38de09bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:25.372174 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-a17afffe-480e-425d-9f3d-b4d38de09bc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:19:25.372174 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 19 15:19:25.372174 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: Aug 19 15:19:25.373108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17afffe-480e-425d-9f3d-b4d38de09bc8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:25.386086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 759/1500] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:19:25.391208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:25.392785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:25.428247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb108ece-145a-432f-9da0-3d75b6bc63b9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:25.539302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8052cd0-9847-440a-a2ed-d5db888b8ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:25.539959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8052cd0-9847-440a-a2ed-d5db888b8ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:25.540233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8052cd0-9847-440a-a2ed-d5db888b8ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:25.540532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8052cd0-9847-440a-a2ed-d5db888b8ea4 admin admin] PATH_INFO: `/v3/projects/4de5d1bec67b4b189785add64cf6ff73/users/438d8e9a13184a0cbf29c7757c53d0ba/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:25.565164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8052cd0-9847-440a-a2ed-d5db888b8ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de5d1bec67b4b189785add64cf6ff73, user_id=438d8e9a13184a0cbf29c7757c53d0ba, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:25.566212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-c8052cd0-9847-440a-a2ed-d5db888b8ea4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:19:25.566212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Aug 19 15:19:25.566212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: Aug 19 15:19:25.568147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8052cd0-9847-440a-a2ed-d5db888b8ea4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:25.600102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 742/1501] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:19:25 2022] PUT /identity/v3/projects/4de5d1bec67b4b189785add64cf6ff73/users/438d8e9a13184a0cbf29c7757c53d0ba/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:19:25.613114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9897154-2b4c-4cd9-9200-acaa6fe15741 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:25.613398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9897154-2b4c-4cd9-9200-acaa6fe15741 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:25.613674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9897154-2b4c-4cd9-9200-acaa6fe15741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:25.680092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f9897154-2b4c-4cd9-9200-acaa6fe15741 None None] MFA Rules not processed for user `438d8e9a13184a0cbf29c7757c53d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:19:25.681444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9897154-2b4c-4cd9-9200-acaa6fe15741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:25.769165 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 760/1502] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:19:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:19:25.785151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f06ab-ee22-4711-bc40-34b156e0dec2 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:25.785438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f06ab-ee22-4711-bc40-34b156e0dec2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:25.785965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f06ab-ee22-4711-bc40-34b156e0dec2 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:25.787340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 743/1503] 10.222.0.75 () {54 vars in 964 bytes} [Fri Aug 19 15:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:19:25.792929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552fc5-b1aa-4896-a086-48cc91a9b8d3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:25.793219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552fc5-b1aa-4896-a086-48cc91a9b8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:25.793490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a552fc5-b1aa-4896-a086-48cc91a9b8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:25.827657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:25.998429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2a552fc5-b1aa-4896-a086-48cc91a9b8d3 None None] MFA Rules not processed for user `a6b4d6a93c7e49009aca527e281b5c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:19:25.999669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2a552fc5-b1aa-4896-a086-48cc91a9b8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:26.079507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 761/1504] 10.222.0.75 () {58 vars in 1061 bytes} [Fri Aug 19 15:19:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:19:26.088977 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f06ab-ee22-4711-bc40-34b156e0dec2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:26.091072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1c0f06ab-ee22-4711-bc40-34b156e0dec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:26.153082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:26.211410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f06ab-ee22-4711-bc40-34b156e0dec2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:26.294925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:26.295532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:26.295813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:26.296085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:26.297734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:26.402771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:26.403629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:26.406653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:26.422480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 744/1505] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:19:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:19:26.468695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b822e0-f1a8-4fa6-9eb8-10495fa40a48 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:26.469143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b822e0-f1a8-4fa6-9eb8-10495fa40a48 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:26.469261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b822e0-f1a8-4fa6-9eb8-10495fa40a48 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:26.470491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 762/1506] 10.222.0.75 () {52 vars in 830 bytes} [Fri Aug 19 15:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:19:26.474172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:26.475410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:26.580439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674a8ed-e886-446d-8c56-448943a7c651 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:26.684175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:26.684836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:26.685116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:26.685393 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] PATH_INFO: `/v3/users/438d8e9a13184a0cbf29c7757c53d0ba` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:26.700469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438d8e9a13184a0cbf29c7757c53d0ba)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:26.701515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:19:26.701515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Aug 19 15:19:26.701515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: Aug 19 15:19:26.702752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:26.713000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:26.774859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438d8e9a13184a0cbf29c7757c53d0ba'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:26.789932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438d8e9a13184a0cbf29c7757c53d0ba'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:26.790510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438d8e9a13184a0cbf29c7757c53d0ba'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:26.815915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438d8e9a13184a0cbf29c7757c53d0ba'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:26.826595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438d8e9a13184a0cbf29c7757c53d0ba'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:26.832985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 745/1507] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:19:26 2022] DELETE /identity/v3/users/438d8e9a13184a0cbf29c7757c53d0ba => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:19:26.837633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542102b-aec7-4414-8b09-540e20de9b5e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:26.838019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542102b-aec7-4414-8b09-540e20de9b5e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:26.838295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542102b-aec7-4414-8b09-540e20de9b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:26.897826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a542102b-aec7-4414-8b09-540e20de9b5e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:19:26.899055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a542102b-aec7-4414-8b09-540e20de9b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:26.991092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:27.006527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 763/1508] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:19:27.018657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:27.020092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:27.138289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db8b6b-4ed9-497d-b945-51bb45972c19 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:27.218966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:27.243087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74029e-2655-4773-b559-7171f3c5cd28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:27.243653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74029e-2655-4773-b559-7171f3c5cd28 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:27.243933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74029e-2655-4773-b559-7171f3c5cd28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:27.244214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74029e-2655-4773-b559-7171f3c5cd28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:27.246099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-da74029e-2655-4773-b559-7171f3c5cd28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:27.372111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74029e-2655-4773-b559-7171f3c5cd28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:27.373655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74029e-2655-4773-b559-7171f3c5cd28 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:27.395671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 746/1509] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:19:27.735961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542102b-aec7-4414-8b09-540e20de9b5e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:19:27.737255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a542102b-aec7-4414-8b09-540e20de9b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:19:27.751113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542102b-aec7-4414-8b09-540e20de9b5e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:19:27.850240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:19:27.850874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:19:27.851152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:19:27.851427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] PATH_INFO: `/v3/projects/4de5d1bec67b4b189785add64cf6ff73` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:19:27.862406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de5d1bec67b4b189785add64cf6ff73)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:19:27.863438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:19:27.863438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 19 15:19:27.863438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: Aug 19 15:19:27.864669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:19:27.923040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de5d1bec67b4b189785add64cf6ff73', 'request_id': 'req-2be83f2b-785a-444d-a357-4174a6bdda96'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:27.929122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de5d1bec67b4b189785add64cf6ff73', 'request_id': 'req-2be83f2b-785a-444d-a357-4174a6bdda96'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:27.940209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de5d1bec67b4b189785add64cf6ff73', 'request_id': 'req-2be83f2b-785a-444d-a357-4174a6bdda96'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:27.970375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] The token cache is being invalidate because project 4de5d1bec67b4b189785add64cf6ff73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:19:27.970665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:27.971096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2be83f2b-785a-444d-a357-4174a6bdda96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:19:27.972542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 764/1510] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:19:27 2022] DELETE /identity/v3/projects/4de5d1bec67b4b189785add64cf6ff73 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:19:37.393308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=62 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:19:37.968432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:21:00.773239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f4b1d-e64d-4018-bc19-e42cfd2b6ed3 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:00.773895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f4b1d-e64d-4018-bc19-e42cfd2b6ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:00.773895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f4b1d-e64d-4018-bc19-e42cfd2b6ed3 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:00.775244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:21:00.780920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea97af-1a3d-4622-b1be-2803436ad17d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:00.781360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea97af-1a3d-4622-b1be-2803436ad17d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:00.781475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea97af-1a3d-4622-b1be-2803436ad17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:00.862511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ffea97af-1a3d-4622-b1be-2803436ad17d None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:00.863714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ffea97af-1a3d-4622-b1be-2803436ad17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:00.969949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:21:01.123759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f4b1d-e64d-4018-bc19-e42cfd2b6ed3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:21:01.125884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-837f4b1d-e64d-4018-bc19-e42cfd2b6ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:01.244788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f4b1d-e64d-4018-bc19-e42cfd2b6ed3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:21:01.250467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bff82-5e1e-47ea-8088-b15736f281c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:21:01.251067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bff82-5e1e-47ea-8088-b15736f281c5 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:01.251344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bff82-5e1e-47ea-8088-b15736f281c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:01.251612 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bff82-5e1e-47ea-8088-b15736f281c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:01.259692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bff82-5e1e-47ea-8088-b15736f281c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:21:01.261447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bff82-5e1e-47ea-8088-b15736f281c5 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:21:01.270574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 748/1513] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:21:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:21:02.116455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329b6bf-498e-473e-a597-220e6d3fb293 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:02.117023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329b6bf-498e-473e-a597-220e6d3fb293 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:02.117023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329b6bf-498e-473e-a597-220e6d3fb293 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:02.118338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:21:02.123722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e488b6-1015-4e16-b45a-08a13287d895 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:02.124197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e488b6-1015-4e16-b45a-08a13287d895 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:02.124314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e488b6-1015-4e16-b45a-08a13287d895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:02.144292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-02e488b6-1015-4e16-b45a-08a13287d895 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:02.145527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-02e488b6-1015-4e16-b45a-08a13287d895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:02.153749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:21:02.310936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329b6bf-498e-473e-a597-220e6d3fb293 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:21:02.326081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329b6bf-498e-473e-a597-220e6d3fb293 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:21:02.331657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd0781-8785-4646-a4f7-725b66329c31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:21:02.332261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd0781-8785-4646-a4f7-725b66329c31 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:02.332574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd0781-8785-4646-a4f7-725b66329c31 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:02.332843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd0781-8785-4646-a4f7-725b66329c31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:02.342266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd0781-8785-4646-a4f7-725b66329c31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:21:02.343900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd0781-8785-4646-a4f7-725b66329c31 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:21:02.353804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 767/1516] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:21:02.849164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075829e-e091-4fcb-b0ac-6bb961fd4b6c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:02.849451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075829e-e091-4fcb-b0ac-6bb961fd4b6c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:02.849720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075829e-e091-4fcb-b0ac-6bb961fd4b6c None None] PATH_INFO: `/v3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:02.851008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 750/1517] 10.222.0.52 () {54 vars in 925 bytes} [Fri Aug 19 15:21:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:21:02.857996 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e22c2-b82e-4c2e-98fe-831234cb4e6a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:02.858279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e22c2-b82e-4c2e-98fe-831234cb4e6a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:02.858552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e22c2-b82e-4c2e-98fe-831234cb4e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:02.916425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b07e22c2-b82e-4c2e-98fe-831234cb4e6a None None] MFA Rules not processed for user `bc2920aac4c046aca75473158ec204cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:02.918308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b07e22c2-b82e-4c2e-98fe-831234cb4e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:02.934779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:21:02.992494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 768/1518] 10.222.0.52 () {58 vars in 1015 bytes} [Fri Aug 19 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:21:07.026836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8e22e-fc87-419b-b272-ae2010c5883c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:07.027406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8e22e-fc87-419b-b272-ae2010c5883c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:07.027406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8e22e-fc87-419b-b272-ae2010c5883c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:07.028767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 751/1519] 10.222.0.75 () {54 vars in 961 bytes} [Fri Aug 19 15:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:21:07.034096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb7c8f-8bc0-41a8-9960-2ca7c86e61ad None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:07.034536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb7c8f-8bc0-41a8-9960-2ca7c86e61ad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:07.034649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb7c8f-8bc0-41a8-9960-2ca7c86e61ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:07.055433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ccfb7c8f-8bc0-41a8-9960-2ca7c86e61ad None None] MFA Rules not processed for user `bc2920aac4c046aca75473158ec204cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:07.056709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ccfb7c8f-8bc0-41a8-9960-2ca7c86e61ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:07.066180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 769/1520] 10.222.0.75 () {58 vars in 1058 bytes} [Fri Aug 19 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:21:07.074522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8e22e-fc87-419b-b272-ae2010c5883c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:21:07.087664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8e22e-fc87-419b-b272-ae2010c5883c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:21:07.093569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a130de6-a013-4680-84f7-861c671facce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:21:07.094158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a130de6-a013-4680-84f7-861c671facce service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:07.094438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a130de6-a013-4680-84f7-861c671facce service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:07.094711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a130de6-a013-4680-84f7-861c671facce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:07.103021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a130de6-a013-4680-84f7-861c671facce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:21:07.104764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a130de6-a013-4680-84f7-861c671facce service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:21:07.114588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 752/1521] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:21:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:21:13.602209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f320b-f5b1-451e-9495-39df07857b58 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:13.602749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f320b-f5b1-451e-9495-39df07857b58 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:13.602749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f320b-f5b1-451e-9495-39df07857b58 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:13.604066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:21:13.609749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f329db-de80-4db2-b8e5-611a1b1eac22 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:13.610195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f329db-de80-4db2-b8e5-611a1b1eac22 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:13.610291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f329db-de80-4db2-b8e5-611a1b1eac22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:13.630134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-38f329db-de80-4db2-b8e5-611a1b1eac22 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:13.631328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-38f329db-de80-4db2-b8e5-611a1b1eac22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:13.639467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:21:13.797912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f320b-f5b1-451e-9495-39df07857b58 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:21:13.813497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f320b-f5b1-451e-9495-39df07857b58 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:21:13.819077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadbf2e-1de1-4e0e-9523-19ca858f9e25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:21:13.819675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadbf2e-1de1-4e0e-9523-19ca858f9e25 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:13.819956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadbf2e-1de1-4e0e-9523-19ca858f9e25 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:13.820229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadbf2e-1de1-4e0e-9523-19ca858f9e25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:13.828460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadbf2e-1de1-4e0e-9523-19ca858f9e25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:21:13.830059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadbf2e-1de1-4e0e-9523-19ca858f9e25 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:21:13.839376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 771/1524] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:21:14.668004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9d580-115c-423f-a05d-f2f230400513 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:14.668603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9d580-115c-423f-a05d-f2f230400513 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:14.668603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9d580-115c-423f-a05d-f2f230400513 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:14.669959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 15:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:21:14.675232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b99a41-779b-472d-ba6a-8b593b8748f9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:14.675684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b99a41-779b-472d-ba6a-8b593b8748f9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:14.675840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b99a41-779b-472d-ba6a-8b593b8748f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:14.695365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-81b99a41-779b-472d-ba6a-8b593b8748f9 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:14.696590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-81b99a41-779b-472d-ba6a-8b593b8748f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:14.704410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:21:14.861410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9d580-115c-423f-a05d-f2f230400513 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:21:14.876056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9d580-115c-423f-a05d-f2f230400513 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:21:14.881508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c886e-1f61-4996-bea2-52ba84ee26ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:21:14.882096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c886e-1f61-4996-bea2-52ba84ee26ce service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:14.882373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c886e-1f61-4996-bea2-52ba84ee26ce service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:14.882649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c886e-1f61-4996-bea2-52ba84ee26ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:14.891016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c886e-1f61-4996-bea2-52ba84ee26ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:21:14.892626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c886e-1f61-4996-bea2-52ba84ee26ce service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:21:14.902581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 755/1527] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:21:18.572835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f53f3-70e1-427b-b6d7-5dd3a82e2e1d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:18.573980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f53f3-70e1-427b-b6d7-5dd3a82e2e1d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:18.574171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f53f3-70e1-427b-b6d7-5dd3a82e2e1d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:18.575523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:21:18.580971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15be78d-cf0f-4b08-8f0a-320d6a21c694 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:18.581394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15be78d-cf0f-4b08-8f0a-320d6a21c694 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:18.581523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15be78d-cf0f-4b08-8f0a-320d6a21c694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:18.601172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d15be78d-cf0f-4b08-8f0a-320d6a21c694 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:18.602374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d15be78d-cf0f-4b08-8f0a-320d6a21c694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:18.610192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:21:18.901138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f53f3-70e1-427b-b6d7-5dd3a82e2e1d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:21:18.902962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-598f53f3-70e1-427b-b6d7-5dd3a82e2e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:18.981067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f53f3-70e1-427b-b6d7-5dd3a82e2e1d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:21:18.984591 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655760d-4925-48de-ab1d-269f54a67f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:21:18.985221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655760d-4925-48de-ab1d-269f54a67f18 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:18.985498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655760d-4925-48de-ab1d-269f54a67f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:18.985772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655760d-4925-48de-ab1d-269f54a67f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:18.995605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655760d-4925-48de-ab1d-269f54a67f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:21:18.997176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655760d-4925-48de-ab1d-269f54a67f18 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:21:19.006741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 774/1530] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:21:20.939321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a9d97-62a4-4397-9de8-a81a43409fd1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:20.939889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a9d97-62a4-4397-9de8-a81a43409fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:20.939889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a9d97-62a4-4397-9de8-a81a43409fd1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:20.941238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:21:20.946567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c19ae1-0415-4c94-b78b-e05705e772df None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:20.947071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c19ae1-0415-4c94-b78b-e05705e772df None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:20.947178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c19ae1-0415-4c94-b78b-e05705e772df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:20.967352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a1c19ae1-0415-4c94-b78b-e05705e772df None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:20.968619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a1c19ae1-0415-4c94-b78b-e05705e772df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:20.976550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:21:21.269759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a9d97-62a4-4397-9de8-a81a43409fd1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:21:21.281463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a9d97-62a4-4397-9de8-a81a43409fd1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:21:21.285080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618999ac-fc15-43d4-9c79-fea30b5040dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:21:21.285678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618999ac-fc15-43d4-9c79-fea30b5040dc service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:21.286002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618999ac-fc15-43d4-9c79-fea30b5040dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:21.286326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618999ac-fc15-43d4-9c79-fea30b5040dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:21.296477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618999ac-fc15-43d4-9c79-fea30b5040dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:21:21.297917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618999ac-fc15-43d4-9c79-fea30b5040dc service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:21:21.307837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 758/1533] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:21:24.983305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da0b91-747e-4c34-80b6-914dbea94e95 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:24.983925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da0b91-747e-4c34-80b6-914dbea94e95 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:24.983925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da0b91-747e-4c34-80b6-914dbea94e95 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:24.985245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:21:24.990530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32e332-eaac-4e82-a937-7cd4c7179f71 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:24.991046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32e332-eaac-4e82-a937-7cd4c7179f71 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:24.991165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32e332-eaac-4e82-a937-7cd4c7179f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:25.011223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7d32e332-eaac-4e82-a937-7cd4c7179f71 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:25.012680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7d32e332-eaac-4e82-a937-7cd4c7179f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:25.021180 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:21:25.593211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da0b91-747e-4c34-80b6-914dbea94e95 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:21:25.594893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-30da0b91-747e-4c34-80b6-914dbea94e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:25.664843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:21:25.683747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da0b91-747e-4c34-80b6-914dbea94e95 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:21:25.687330 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d416c-58c3-44fa-8038-5d7116b21363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:21:25.687936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d416c-58c3-44fa-8038-5d7116b21363 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:25.688213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d416c-58c3-44fa-8038-5d7116b21363 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:25.688523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d416c-58c3-44fa-8038-5d7116b21363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:25.698611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d416c-58c3-44fa-8038-5d7116b21363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:21:25.700069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d416c-58c3-44fa-8038-5d7116b21363 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:21:25.709849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 777/1536] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:21:27.698293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fa889-9122-4a13-9104-a9973945cb4a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:27.699037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fa889-9122-4a13-9104-a9973945cb4a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:27.699037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fa889-9122-4a13-9104-a9973945cb4a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:27.700310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 15:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:21:27.705981 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36304004-b93e-4b7c-964a-3a6e9738252d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:27.706474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36304004-b93e-4b7c-964a-3a6e9738252d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:27.706577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36304004-b93e-4b7c-964a-3a6e9738252d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:27.726850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-36304004-b93e-4b7c-964a-3a6e9738252d None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:21:27.728088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-36304004-b93e-4b7c-964a-3a6e9738252d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:27.736760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 15:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:21:28.311289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fa889-9122-4a13-9104-a9973945cb4a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:21:28.313011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-767fa889-9122-4a13-9104-a9973945cb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:21:28.320514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:21:28.327346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fa889-9122-4a13-9104-a9973945cb4a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:21:28.334194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15d4cc-f2da-4421-9552-5c681582df11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:21:28.334813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15d4cc-f2da-4421-9552-5c681582df11 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:21:28.335095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15d4cc-f2da-4421-9552-5c681582df11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:21:28.335370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15d4cc-f2da-4421-9552-5c681582df11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:21:28.345757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15d4cc-f2da-4421-9552-5c681582df11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:21:28.347219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15d4cc-f2da-4421-9552-5c681582df11 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:21:28.358141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 761/1539] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:21:37.721920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=11 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:21:38.352877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:55.604429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831519f-55c2-429c-bba4-a2303d9bd319 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:55.605114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831519f-55c2-429c-bba4-a2303d9bd319 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:55.605239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831519f-55c2-429c-bba4-a2303d9bd319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:55.612684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9d703-bd90-4973-96ac-0762df4fa289 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:55.613223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9d703-bd90-4973-96ac-0762df4fa289 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:55.613223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9d703-bd90-4973-96ac-0762df4fa289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:55.632557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2831519f-55c2-429c-bba4-a2303d9bd319 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:55.633875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2831519f-55c2-429c-bba4-a2303d9bd319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:55.640970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b0a9d703-bd90-4973-96ac-0762df4fa289 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:55.642171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b0a9d703-bd90-4973-96ac-0762df4fa289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:55.651084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 779/1540] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:55.654755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 762/1541] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:22:55.660791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2332c-7a7f-46b0-9e0c-e0bdd6d0ed18 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:55.661074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2332c-7a7f-46b0-9e0c-e0bdd6d0ed18 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:55.661480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2332c-7a7f-46b0-9e0c-e0bdd6d0ed18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:55.662854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108585f6-e2bd-440a-9632-9d43b32e11f8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:55.663135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108585f6-e2bd-440a-9632-9d43b32e11f8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:55.663408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108585f6-e2bd-440a-9632-9d43b32e11f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:55.694457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-108585f6-e2bd-440a-9632-9d43b32e11f8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:55.695649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-108585f6-e2bd-440a-9632-9d43b32e11f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:55.704219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fed2332c-7a7f-46b0-9e0c-e0bdd6d0ed18 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:55.705464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fed2332c-7a7f-46b0-9e0c-e0bdd6d0ed18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:55.711119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 763/1542] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:22:55.716766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108585f6-e2bd-440a-9632-9d43b32e11f8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:55.732007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108585f6-e2bd-440a-9632-9d43b32e11f8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:55.732227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 780/1543] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:55.736857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2332c-7a7f-46b0-9e0c-e0bdd6d0ed18 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:55.739436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:55.740055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:55.740334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:55.740640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:55.741475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:22:55.741783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:55.742814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:22:55.742814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Aug 19 15:22:55.742814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: Aug 19 15:22:55.743708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:55.747522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2332c-7a7f-46b0-9e0c-e0bdd6d0ed18 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:55.768427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbfdd4-2515-4fce-94ae-5d9c4353822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:55.768840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 764/1544] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:22:55.769134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbfdd4-2515-4fce-94ae-5d9c4353822d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:55.769478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbfdd4-2515-4fce-94ae-5d9c4353822d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:55.769755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbfdd4-2515-4fce-94ae-5d9c4353822d admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:55.770418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbfdd4-2515-4fce-94ae-5d9c4353822d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:22:55.770723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbfdd4-2515-4fce-94ae-5d9c4353822d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:55.773041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dbfdd4-2515-4fce-94ae-5d9c4353822d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:55.777865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:55.799661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5816-5b11-408c-8bb6-02fbf8b438ba admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:55.806015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 781/1545] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:22:55.808688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce9b97-4301-4c83-ac26-537c306a0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:55.809493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbfdd4-2515-4fce-94ae-5d9c4353822d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:55.809700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce9b97-4301-4c83-ac26-537c306a0426 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:55.810030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce9b97-4301-4c83-ac26-537c306a0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:55.810305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce9b97-4301-4c83-ac26-537c306a0426 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:55.810941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce9b97-4301-4c83-ac26-537c306a0426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:55.812942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce9b97-4301-4c83-ac26-537c306a0426 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:55.823037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbfdd4-2515-4fce-94ae-5d9c4353822d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:55.836893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9e83b-f249-4232-9d2d-88067787ee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:55.837518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9e83b-f249-4232-9d2d-88067787ee80 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:55.837794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9e83b-f249-4232-9d2d-88067787ee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:55.838066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9e83b-f249-4232-9d2d-88067787ee80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:55.838706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9e83b-f249-4232-9d2d-88067787ee80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:55.841023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-0bf9e83b-f249-4232-9d2d-88067787ee80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:22:55.841023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 19 15:22:55.841023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: Aug 19 15:22:55.842244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9e83b-f249-4232-9d2d-88067787ee80 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:55.896286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 765/1546] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:22:55 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:22:55.902993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce9b97-4301-4c83-ac26-537c306a0426 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:55.910636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 782/1547] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:22:55 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:22:55.923788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9e83b-f249-4232-9d2d-88067787ee80 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.048118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce9b97-4301-4c83-ac26-537c306a0426 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.055255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9e83b-f249-4232-9d2d-88067787ee80 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.061471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f1331-3814-4c3f-b6d1-4eb87765ec2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.063125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f1331-3814-4c3f-b6d1-4eb87765ec2f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.064142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f1331-3814-4c3f-b6d1-4eb87765ec2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.065117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f1331-3814-4c3f-b6d1-4eb87765ec2f admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.066421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f1331-3814-4c3f-b6d1-4eb87765ec2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.068447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-942f1331-3814-4c3f-b6d1-4eb87765ec2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:22:56.068447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Aug 19 15:22:56.068447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: Aug 19 15:22:56.070185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f1331-3814-4c3f-b6d1-4eb87765ec2f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.080513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6545b9f-b3f2-4353-88c4-4e0a94b84ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.080513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6545b9f-b3f2-4353-88c4-4e0a94b84ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.080513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6545b9f-b3f2-4353-88c4-4e0a94b84ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.080513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6545b9f-b3f2-4353-88c4-4e0a94b84ea1 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.081009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6545b9f-b3f2-4353-88c4-4e0a94b84ea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.081009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6545b9f-b3f2-4353-88c4-4e0a94b84ea1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.121367 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 766/1549] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:22:55 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:56.124589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 783/1549] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:22:55 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:22:56.126491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6081b-2823-44bb-a2d0-ac9c6ae9f8ce None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.126644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e6af8-c7a6-4dd0-b3d6-968e6075ea13 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.126838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6081b-2823-44bb-a2d0-ac9c6ae9f8ce None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.126953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e6af8-c7a6-4dd0-b3d6-968e6075ea13 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.127043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6081b-2823-44bb-a2d0-ac9c6ae9f8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.127139 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e6af8-c7a6-4dd0-b3d6-968e6075ea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.150600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3bc6081b-2823-44bb-a2d0-ac9c6ae9f8ce None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.152235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3bc6081b-2823-44bb-a2d0-ac9c6ae9f8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.152594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1f8e6af8-c7a6-4dd0-b3d6-968e6075ea13 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.153978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f8e6af8-c7a6-4dd0-b3d6-968e6075ea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.162075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 767/1550] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.163761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 784/1551] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.165352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d816d4-e6d4-4760-8fd9-57eaed8bcb5b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.165640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d816d4-e6d4-4760-8fd9-57eaed8bcb5b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.165915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d816d4-e6d4-4760-8fd9-57eaed8bcb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.169402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac37661-b61e-42d9-b3d1-1938b345b7fe None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.169685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac37661-b61e-42d9-b3d1-1938b345b7fe None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.169967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac37661-b61e-42d9-b3d1-1938b345b7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.191858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-83d816d4-e6d4-4760-8fd9-57eaed8bcb5b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.193424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-83d816d4-e6d4-4760-8fd9-57eaed8bcb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.203186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 785/1552] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.203925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d816d4-e6d4-4760-8fd9-57eaed8bcb5b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.207813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-aac37661-b61e-42d9-b3d1-1938b345b7fe None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.209355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-aac37661-b61e-42d9-b3d1-1938b345b7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.214521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d816d4-e6d4-4760-8fd9-57eaed8bcb5b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.219439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 768/1553] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.219625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592585d8-622d-475f-92bd-495e8b685c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.220277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592585d8-622d-475f-92bd-495e8b685c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.220589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592585d8-622d-475f-92bd-495e8b685c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.220863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592585d8-622d-475f-92bd-495e8b685c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.221514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592585d8-622d-475f-92bd-495e8b685c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.223128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592585d8-622d-475f-92bd-495e8b685c04 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.225432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac37661-b61e-42d9-b3d1-1938b345b7fe None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.236159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 786/1554] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:56.237119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac37661-b61e-42d9-b3d1-1938b345b7fe None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.238153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c3990-48db-4706-9bf1-1e7234d4c4a6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.238615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c3990-48db-4706-9bf1-1e7234d4c4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.239458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c3990-48db-4706-9bf1-1e7234d4c4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.243783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad1c43-ba96-4f9e-8584-bb269dddc900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.244509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad1c43-ba96-4f9e-8584-bb269dddc900 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.244791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad1c43-ba96-4f9e-8584-bb269dddc900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.245062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad1c43-ba96-4f9e-8584-bb269dddc900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.246576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad1c43-ba96-4f9e-8584-bb269dddc900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.249740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad1c43-ba96-4f9e-8584-bb269dddc900 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.259217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a56c3990-48db-4706-9bf1-1e7234d4c4a6 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.260453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a56c3990-48db-4706-9bf1-1e7234d4c4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.262948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 769/1555] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:56.264469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22d63b-e830-41bf-bbcd-93ee5d862197 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.264758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22d63b-e830-41bf-bbcd-93ee5d862197 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.265030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22d63b-e830-41bf-bbcd-93ee5d862197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.275926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 787/1556] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.275926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11845a-c65c-4ad4-8422-fafa72e72744 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.275926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11845a-c65c-4ad4-8422-fafa72e72744 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.275926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11845a-c65c-4ad4-8422-fafa72e72744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.293928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ec22d63b-e830-41bf-bbcd-93ee5d862197 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.295163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ec22d63b-e830-41bf-bbcd-93ee5d862197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.303073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2a11845a-c65c-4ad4-8422-fafa72e72744 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.303302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 770/1557] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.304520 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f471216-7315-44b7-8a8c-b52804418907 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.304688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2a11845a-c65c-4ad4-8422-fafa72e72744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.304799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f471216-7315-44b7-8a8c-b52804418907 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.305068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f471216-7315-44b7-8a8c-b52804418907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.314073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 788/1558] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.315446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92a0b-b1bb-4eb6-99c8-c8a6add6fd59 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.315833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92a0b-b1bb-4eb6-99c8-c8a6add6fd59 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.316204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d92a0b-b1bb-4eb6-99c8-c8a6add6fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.326139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3f471216-7315-44b7-8a8c-b52804418907 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.327351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f471216-7315-44b7-8a8c-b52804418907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.335478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 771/1559] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.335735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b2d92a0b-b1bb-4eb6-99c8-c8a6add6fd59 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.336374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f471216-7315-44b7-8a8c-b52804418907 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.336956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b2d92a0b-b1bb-4eb6-99c8-c8a6add6fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.347025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f471216-7315-44b7-8a8c-b52804418907 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.348417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 789/1560] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.349260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d92a0b-b1bb-4eb6-99c8-c8a6add6fd59 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.353322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5123be1-ce0a-402c-9493-bfcb8970d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.353936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5123be1-ce0a-402c-9493-bfcb8970d558 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.354218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5123be1-ce0a-402c-9493-bfcb8970d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.354492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5123be1-ce0a-402c-9493-bfcb8970d558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.355690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5123be1-ce0a-402c-9493-bfcb8970d558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.357513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5123be1-ce0a-402c-9493-bfcb8970d558 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.363479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d92a0b-b1bb-4eb6-99c8-c8a6add6fd59 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.368410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738456e-a46f-4baa-be9e-028552e5b5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.369041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738456e-a46f-4baa-be9e-028552e5b5cc admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.369323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738456e-a46f-4baa-be9e-028552e5b5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.369544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 772/1561] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:56.369641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738456e-a46f-4baa-be9e-028552e5b5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.370290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738456e-a46f-4baa-be9e-028552e5b5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.371022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5123be1-ce0a-402c-9493-bfcb8970d558 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.372477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738456e-a46f-4baa-be9e-028552e5b5cc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.380869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5123be1-ce0a-402c-9493-bfcb8970d558 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.383032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 790/1562] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:56.384139 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942473fb-b458-4293-9db0-b9e14d1c15f1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.384449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942473fb-b458-4293-9db0-b9e14d1c15f1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.384727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942473fb-b458-4293-9db0-b9e14d1c15f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.388698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefef85-77be-4caa-960f-b902014ea92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.389305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefef85-77be-4caa-960f-b902014ea92d admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.389584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefef85-77be-4caa-960f-b902014ea92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.389857 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefef85-77be-4caa-960f-b902014ea92d admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.390506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefef85-77be-4caa-960f-b902014ea92d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:22:56.390815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefef85-77be-4caa-960f-b902014ea92d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.392557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefef85-77be-4caa-960f-b902014ea92d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.404889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-942473fb-b458-4293-9db0-b9e14d1c15f1 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.412618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 773/1563] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:22:56.412834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-942473fb-b458-4293-9db0-b9e14d1c15f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.412955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefef85-77be-4caa-960f-b902014ea92d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.419134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefef85-77be-4caa-960f-b902014ea92d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.422408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 791/1564] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.424400 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62f5ae-26c3-4aca-9255-1c2adbc80627 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.424881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62f5ae-26c3-4aca-9255-1c2adbc80627 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.425352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62f5ae-26c3-4aca-9255-1c2adbc80627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.430414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b59f88-92d8-4af9-bb68-f41d9a416f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.431025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b59f88-92d8-4af9-bb68-f41d9a416f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.431302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b59f88-92d8-4af9-bb68-f41d9a416f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.431566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b59f88-92d8-4af9-bb68-f41d9a416f57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.432214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b59f88-92d8-4af9-bb68-f41d9a416f57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:22:56.432543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b59f88-92d8-4af9-bb68-f41d9a416f57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.434974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b59f88-92d8-4af9-bb68-f41d9a416f57 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.454050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 774/1565] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:22:56.454050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b59f88-92d8-4af9-bb68-f41d9a416f57 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.455591 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dd62f5ae-26c3-4aca-9255-1c2adbc80627 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.456876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dd62f5ae-26c3-4aca-9255-1c2adbc80627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.461408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b59f88-92d8-4af9-bb68-f41d9a416f57 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.464919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 792/1566] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.465721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62f5ae-26c3-4aca-9255-1c2adbc80627 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.470072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee72a3f-20ae-440e-aba2-fd9fb09881a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.470681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee72a3f-20ae-440e-aba2-fd9fb09881a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.470957 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee72a3f-20ae-440e-aba2-fd9fb09881a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.471224 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee72a3f-20ae-440e-aba2-fd9fb09881a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.471872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee72a3f-20ae-440e-aba2-fd9fb09881a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:22:56.472285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee72a3f-20ae-440e-aba2-fd9fb09881a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.473994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee72a3f-20ae-440e-aba2-fd9fb09881a0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.475808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62f5ae-26c3-4aca-9255-1c2adbc80627 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.482056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0aaf7-9c80-42d7-8982-dd3c942cc1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.482707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0aaf7-9c80-42d7-8982-dd3c942cc1bf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.482864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0aaf7-9c80-42d7-8982-dd3c942cc1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.483146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0aaf7-9c80-42d7-8982-dd3c942cc1bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.483792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0aaf7-9c80-42d7-8982-dd3c942cc1bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:22:56.484587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0aaf7-9c80-42d7-8982-dd3c942cc1bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.486210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0aaf7-9c80-42d7-8982-dd3c942cc1bf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.487547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 775/1567] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:22:56.494183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee72a3f-20ae-440e-aba2-fd9fb09881a0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.497029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:56.499889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 793/1568] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:22:56.500805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0aaf7-9c80-42d7-8982-dd3c942cc1bf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.501434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee72a3f-20ae-440e-aba2-fd9fb09881a0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.506084 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878efacb-2343-4fcf-9b1e-7ab43615913c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.506746 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878efacb-2343-4fcf-9b1e-7ab43615913c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.507115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878efacb-2343-4fcf-9b1e-7ab43615913c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.507426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878efacb-2343-4fcf-9b1e-7ab43615913c admin admin] PATH_INFO: `/v3/projects/2159758dea7c4b23a31fa18fde2fb863/users/99b2e057a01247d4960d1d7895bb465d/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.512613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0aaf7-9c80-42d7-8982-dd3c942cc1bf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.517167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55346639-e434-4e81-8de4-f8adbab14593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.517825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55346639-e434-4e81-8de4-f8adbab14593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.518106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55346639-e434-4e81-8de4-f8adbab14593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.518380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55346639-e434-4e81-8de4-f8adbab14593 admin admin] PATH_INFO: `/v3/projects/8ede055158e24b7c9a935b833e33b89a/users/dc2bbdceb12641e6a0595e68f7cefd63/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.524893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878efacb-2343-4fcf-9b1e-7ab43615913c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2159758dea7c4b23a31fa18fde2fb863, user_id=99b2e057a01247d4960d1d7895bb465d, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.528088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878efacb-2343-4fcf-9b1e-7ab43615913c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.534091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55346639-e434-4e81-8de4-f8adbab14593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ede055158e24b7c9a935b833e33b89a, user_id=dc2bbdceb12641e6a0595e68f7cefd63, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.535283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-55346639-e434-4e81-8de4-f8adbab14593 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:22:56.535283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 19 15:22:56.535283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: Aug 19 15:22:56.537610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55346639-e434-4e81-8de4-f8adbab14593 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.541075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:56.541358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 776/1569] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:56 2022] PUT /identity/v3/projects/2159758dea7c4b23a31fa18fde2fb863/users/99b2e057a01247d4960d1d7895bb465d/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:56.542710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878efacb-2343-4fcf-9b1e-7ab43615913c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.569351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 794/1570] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:56 2022] PUT /identity/v3/projects/8ede055158e24b7c9a935b833e33b89a/users/dc2bbdceb12641e6a0595e68f7cefd63/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:56.571136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c737412-02d7-401a-9764-9a8d9caa9486 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.571475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c737412-02d7-401a-9764-9a8d9caa9486 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.571806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c737412-02d7-401a-9764-9a8d9caa9486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.591466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5c737412-02d7-401a-9764-9a8d9caa9486 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.592795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c737412-02d7-401a-9764-9a8d9caa9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.607985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878efacb-2343-4fcf-9b1e-7ab43615913c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.659973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 795/1571] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.660912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c737412-02d7-401a-9764-9a8d9caa9486 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.672176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c737412-02d7-401a-9764-9a8d9caa9486 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.679104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0a9f1-e223-4817-9299-cf6595f187a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.679714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0a9f1-e223-4817-9299-cf6595f187a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.679830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fd705-3a58-44f6-851e-c388f5ae6c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.679989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0a9f1-e223-4817-9299-cf6595f187a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.680284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0a9f1-e223-4817-9299-cf6595f187a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.680409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fd705-3a58-44f6-851e-c388f5ae6c28 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.680682 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fd705-3a58-44f6-851e-c388f5ae6c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.680978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fd705-3a58-44f6-851e-c388f5ae6c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.681081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0a9f1-e223-4817-9299-cf6595f187a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.681602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fd705-3a58-44f6-851e-c388f5ae6c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.682975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0a9f1-e223-4817-9299-cf6595f187a6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.683961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fd705-3a58-44f6-851e-c388f5ae6c28 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.713859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 777/1572] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:56.713859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0a9f1-e223-4817-9299-cf6595f187a6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.714682 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 796/1573] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:56.715397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fd705-3a58-44f6-851e-c388f5ae6c28 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.774906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0a9f1-e223-4817-9299-cf6595f187a6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.780211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576955f-19cb-49be-bf32-13f287c8ff8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.780845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576955f-19cb-49be-bf32-13f287c8ff8b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.781125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576955f-19cb-49be-bf32-13f287c8ff8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.781395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576955f-19cb-49be-bf32-13f287c8ff8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.781492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fd705-3a58-44f6-851e-c388f5ae6c28 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.782039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576955f-19cb-49be-bf32-13f287c8ff8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:22:56.782346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576955f-19cb-49be-bf32-13f287c8ff8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.783951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576955f-19cb-49be-bf32-13f287c8ff8b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.786205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0d240-fc76-4384-894d-05d11f35cbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.786808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0d240-fc76-4384-894d-05d11f35cbd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.787081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0d240-fc76-4384-894d-05d11f35cbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.787351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0d240-fc76-4384-894d-05d11f35cbd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.787972 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0d240-fc76-4384-894d-05d11f35cbd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.790234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0d240-fc76-4384-894d-05d11f35cbd7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.797434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 778/1574] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:22:56.800145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576955f-19cb-49be-bf32-13f287c8ff8b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.811966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576955f-19cb-49be-bf32-13f287c8ff8b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.816817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2c44-2067-4ce8-97a0-c59ad5c42194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.817363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2c44-2067-4ce8-97a0-c59ad5c42194 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.817641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2c44-2067-4ce8-97a0-c59ad5c42194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.817913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2c44-2067-4ce8-97a0-c59ad5c42194 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.818563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b2c44-2067-4ce8-97a0-c59ad5c42194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.822307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 797/1575] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:56.822307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7a3b7-44a7-4979-9506-11fb29f9e4aa None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.822307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7a3b7-44a7-4979-9506-11fb29f9e4aa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.822650 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b2c44-2067-4ce8-97a0-c59ad5c42194 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.822744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7a3b7-44a7-4979-9506-11fb29f9e4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.825744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:56.853975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 779/1576] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:56.854182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f4f7a3b7-44a7-4979-9506-11fb29f9e4aa None None] MFA Rules not processed for user `99b2e057a01247d4960d1d7895bb465d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.854330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc62e92-b833-40ec-9ff8-6be59c6acd8c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.854330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc62e92-b833-40ec-9ff8-6be59c6acd8c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.854504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f4f7a3b7-44a7-4979-9506-11fb29f9e4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.854613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc62e92-b833-40ec-9ff8-6be59c6acd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.871653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ddc62e92-b833-40ec-9ff8-6be59c6acd8c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:56.872886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ddc62e92-b833-40ec-9ff8-6be59c6acd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:56.904940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:56.924217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 798/1577] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.925255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7a3b7-44a7-4979-9506-11fb29f9e4aa None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.939077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 780/1578] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:56.939932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc62e92-b833-40ec-9ff8-6be59c6acd8c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.951041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc62e92-b833-40ec-9ff8-6be59c6acd8c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.955950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f62410-6ac4-40c0-9d39-8c15c0658056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.956609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f62410-6ac4-40c0-9d39-8c15c0658056 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.956887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f62410-6ac4-40c0-9d39-8c15c0658056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.957344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f62410-6ac4-40c0-9d39-8c15c0658056 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.957981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f62410-6ac4-40c0-9d39-8c15c0658056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.959918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f62410-6ac4-40c0-9d39-8c15c0658056 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:56.981406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 781/1579] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:56.982399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f62410-6ac4-40c0-9d39-8c15c0658056 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:56.992174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f62410-6ac4-40c0-9d39-8c15c0658056 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.993949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7a3b7-44a7-4979-9506-11fb29f9e4aa None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:56.997418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc7808-bc1b-48f3-b478-2a5b58209263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:56.998008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc7808-bc1b-48f3-b478-2a5b58209263 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:56.998284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc7808-bc1b-48f3-b478-2a5b58209263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:56.998604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc7808-bc1b-48f3-b478-2a5b58209263 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:56.999162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc7808-bc1b-48f3-b478-2a5b58209263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:56.999855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2e8df-669b-4985-bee1-63454d2cecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.000482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2e8df-669b-4985-bee1-63454d2cecb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.000919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2e8df-669b-4985-bee1-63454d2cecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.001144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc7808-bc1b-48f3-b478-2a5b58209263 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.001251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2e8df-669b-4985-bee1-63454d2cecb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.001829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2e8df-669b-4985-bee1-63454d2cecb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:22:57.002129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2e8df-669b-4985-bee1-63454d2cecb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.003722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2e8df-669b-4985-bee1-63454d2cecb9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.016483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 799/1580] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:22:57.017140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2e8df-669b-4985-bee1-63454d2cecb9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.030097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2e8df-669b-4985-bee1-63454d2cecb9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.031409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 782/1581] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:22:56 2022] POST /identity/v3/users => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:57.032285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc7808-bc1b-48f3-b478-2a5b58209263 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.035077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35e72f-5219-48a3-abd9-16f6a5bed38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.035711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35e72f-5219-48a3-abd9-16f6a5bed38a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.035989 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35e72f-5219-48a3-abd9-16f6a5bed38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.036260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35e72f-5219-48a3-abd9-16f6a5bed38a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.036928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35e72f-5219-48a3-abd9-16f6a5bed38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.038855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35e72f-5219-48a3-abd9-16f6a5bed38a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.042966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc7808-bc1b-48f3-b478-2a5b58209263 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.048658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021c681-56f3-48c0-b41a-75156c23ba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.049356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021c681-56f3-48c0-b41a-75156c23ba58 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.049647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021c681-56f3-48c0-b41a-75156c23ba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.049927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021c681-56f3-48c0-b41a-75156c23ba58 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.050544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021c681-56f3-48c0-b41a-75156c23ba58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.052491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021c681-56f3-48c0-b41a-75156c23ba58 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.069964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 800/1582] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:22:57 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:57.070871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35e72f-5219-48a3-abd9-16f6a5bed38a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.078853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 783/1583] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:22:57 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:57.078853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021c681-56f3-48c0-b41a-75156c23ba58 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.086760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021c681-56f3-48c0-b41a-75156c23ba58 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.134458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35e72f-5219-48a3-abd9-16f6a5bed38a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.140025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c8fe7-37b9-4b28-b453-1d30f0eeb624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.140737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c8fe7-37b9-4b28-b453-1d30f0eeb624 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.141018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c8fe7-37b9-4b28-b453-1d30f0eeb624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.141298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c8fe7-37b9-4b28-b453-1d30f0eeb624 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.142813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284d00e-946e-4a29-84cf-d53d72efb232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.142936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c8fe7-37b9-4b28-b453-1d30f0eeb624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.143421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284d00e-946e-4a29-84cf-d53d72efb232 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.143702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284d00e-946e-4a29-84cf-d53d72efb232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.143977 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284d00e-946e-4a29-84cf-d53d72efb232 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.144632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284d00e-946e-4a29-84cf-d53d72efb232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.144915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c8fe7-37b9-4b28-b453-1d30f0eeb624 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.146535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284d00e-946e-4a29-84cf-d53d72efb232 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.167636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 801/1584] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:22:57 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:57.168504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c8fe7-37b9-4b28-b453-1d30f0eeb624 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.170123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-694c8fe7-37b9-4b28-b453-1d30f0eeb624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:57.180756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 784/1585] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:22:57 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:57.180756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284d00e-946e-4a29-84cf-d53d72efb232 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.202429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:57.227510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c8fe7-37b9-4b28-b453-1d30f0eeb624 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.232231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:57.246185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284d00e-946e-4a29-84cf-d53d72efb232 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.250943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0e7d2-5858-4e93-8918-0247beead34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.251575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0e7d2-5858-4e93-8918-0247beead34c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.251858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0e7d2-5858-4e93-8918-0247beead34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.252136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0e7d2-5858-4e93-8918-0247beead34c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.252818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0e7d2-5858-4e93-8918-0247beead34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.254439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0e7d2-5858-4e93-8918-0247beead34c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.267771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 785/1586] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.267771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0e7d2-5858-4e93-8918-0247beead34c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.275342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0e7d2-5858-4e93-8918-0247beead34c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.279928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeec973-fefd-4bf2-a45b-a921f938542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.280549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeec973-fefd-4bf2-a45b-a921f938542f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.280838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeec973-fefd-4bf2-a45b-a921f938542f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.281109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeec973-fefd-4bf2-a45b-a921f938542f admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.281736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeec973-fefd-4bf2-a45b-a921f938542f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.283783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeec973-fefd-4bf2-a45b-a921f938542f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.284505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5ea4b-1db4-407e-ab3b-eab6216f2ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.285209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5ea4b-1db4-407e-ab3b-eab6216f2ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.285490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5ea4b-1db4-407e-ab3b-eab6216f2ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.285767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5ea4b-1db4-407e-ab3b-eab6216f2ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.296578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5ea4b-1db4-407e-ab3b-eab6216f2ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.298024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c5ea4b-1db4-407e-ab3b-eab6216f2ee6 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.307685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 802/1587] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:22:57.308951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5ea4b-1db4-407e-ab3b-eab6216f2ee6 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.313771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 786/1588] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:22:57 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:57.314575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeec973-fefd-4bf2-a45b-a921f938542f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.318650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5ea4b-1db4-407e-ab3b-eab6216f2ee6 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.381566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a3433-8d3d-40b1-bc75-435ae22eb02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.382323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a3433-8d3d-40b1-bc75-435ae22eb02d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.382697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a3433-8d3d-40b1-bc75-435ae22eb02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.383066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a3433-8d3d-40b1-bc75-435ae22eb02d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.383790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a3433-8d3d-40b1-bc75-435ae22eb02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.385511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a3433-8d3d-40b1-bc75-435ae22eb02d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.385656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeec973-fefd-4bf2-a45b-a921f938542f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.391723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437eba-3083-45b3-b5b3-926e9733bd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.392603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437eba-3083-45b3-b5b3-926e9733bd00 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.392960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437eba-3083-45b3-b5b3-926e9733bd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.393299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437eba-3083-45b3-b5b3-926e9733bd00 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.394012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db437eba-3083-45b3-b5b3-926e9733bd00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.396320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db437eba-3083-45b3-b5b3-926e9733bd00 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.400380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 803/1589] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.400380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a3433-8d3d-40b1-bc75-435ae22eb02d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.412821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a3433-8d3d-40b1-bc75-435ae22eb02d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.420971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ccf85-18be-4bf0-a899-76431bd4d407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.422114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ccf85-18be-4bf0-a899-76431bd4d407 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.422446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ccf85-18be-4bf0-a899-76431bd4d407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.422721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ccf85-18be-4bf0-a899-76431bd4d407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.423433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ccf85-18be-4bf0-a899-76431bd4d407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.425115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ccf85-18be-4bf0-a899-76431bd4d407 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.430268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 787/1590] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:22:57 2022] POST /identity/v3/users => generated 303 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:57.430268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437eba-3083-45b3-b5b3-926e9733bd00 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.438549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 804/1591] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.439729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ccf85-18be-4bf0-a899-76431bd4d407 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.441614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437eba-3083-45b3-b5b3-926e9733bd00 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.446887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf59073-0bbf-4d97-8a9d-e354296bcf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.448235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf59073-0bbf-4d97-8a9d-e354296bcf2a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.449256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf59073-0bbf-4d97-8a9d-e354296bcf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.449660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf59073-0bbf-4d97-8a9d-e354296bcf2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.450328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf59073-0bbf-4d97-8a9d-e354296bcf2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.451588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ccf85-18be-4bf0-a899-76431bd4d407 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.452051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf59073-0bbf-4d97-8a9d-e354296bcf2a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.457396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09d77e-47fd-45c5-a929-4139aee40bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.458012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09d77e-47fd-45c5-a929-4139aee40bcc admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.458358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09d77e-47fd-45c5-a929-4139aee40bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.458694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09d77e-47fd-45c5-a929-4139aee40bcc admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.459390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09d77e-47fd-45c5-a929-4139aee40bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.461774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09d77e-47fd-45c5-a929-4139aee40bcc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.468239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 788/1592] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.469536 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf59073-0bbf-4d97-8a9d-e354296bcf2a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.483496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf59073-0bbf-4d97-8a9d-e354296bcf2a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.487501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 805/1593] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:22:57 2022] POST /identity/v3/users => generated 294 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:57.488185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09d77e-47fd-45c5-a929-4139aee40bcc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.489163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee492fe3-aac5-4565-aad5-76f5d337c53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.490038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee492fe3-aac5-4565-aad5-76f5d337c53b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.490506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee492fe3-aac5-4565-aad5-76f5d337c53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.490967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee492fe3-aac5-4565-aad5-76f5d337c53b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.491901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee492fe3-aac5-4565-aad5-76f5d337c53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.494084 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee492fe3-aac5-4565-aad5-76f5d337c53b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.502390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09d77e-47fd-45c5-a929-4139aee40bcc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.514662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 789/1594] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.514662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee492fe3-aac5-4565-aad5-76f5d337c53b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.523318 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d2835-9e62-48d6-a7c0-ec73fc52f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.524193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee492fe3-aac5-4565-aad5-76f5d337c53b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.524337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d2835-9e62-48d6-a7c0-ec73fc52f265 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.524781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d2835-9e62-48d6-a7c0-ec73fc52f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.525337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d2835-9e62-48d6-a7c0-ec73fc52f265 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.526559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d2835-9e62-48d6-a7c0-ec73fc52f265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.529126 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c2491-40a4-481d-93d9-1f944a7bf6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.529645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d2835-9e62-48d6-a7c0-ec73fc52f265 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.529817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2491-40a4-481d-93d9-1f944a7bf6aa admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.530100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2491-40a4-481d-93d9-1f944a7bf6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.530369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2491-40a4-481d-93d9-1f944a7bf6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.531013 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c2491-40a4-481d-93d9-1f944a7bf6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.532645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c2491-40a4-481d-93d9-1f944a7bf6aa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.543844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 790/1595] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.543844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c2491-40a4-481d-93d9-1f944a7bf6aa admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.554980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c2491-40a4-481d-93d9-1f944a7bf6aa admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.560502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243d625-1955-4467-a668-aba1b0059eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.561304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243d625-1955-4467-a668-aba1b0059eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.561408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243d625-1955-4467-a668-aba1b0059eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.561665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243d625-1955-4467-a668-aba1b0059eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.562308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243d625-1955-4467-a668-aba1b0059eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.567088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243d625-1955-4467-a668-aba1b0059eb1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.567277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 806/1596] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:22:57 2022] POST /identity/v3/users => generated 282 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:22:57.567277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d2835-9e62-48d6-a7c0-ec73fc52f265 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.578273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 791/1597] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.578273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243d625-1955-4467-a668-aba1b0059eb1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.581881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d2835-9e62-48d6-a7c0-ec73fc52f265 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.586024 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243d625-1955-4467-a668-aba1b0059eb1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.586788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84151838-de37-444b-82e9-1695838a03a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.587746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84151838-de37-444b-82e9-1695838a03a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.587871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84151838-de37-444b-82e9-1695838a03a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.588311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84151838-de37-444b-82e9-1695838a03a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.589114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84151838-de37-444b-82e9-1695838a03a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.591191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84151838-de37-444b-82e9-1695838a03a6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.591962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc2d70-8c91-41e3-8bfa-6ce4bedf92e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.592487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc2d70-8c91-41e3-8bfa-6ce4bedf92e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.592842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc2d70-8c91-41e3-8bfa-6ce4bedf92e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.593116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc2d70-8c91-41e3-8bfa-6ce4bedf92e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.593759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc2d70-8c91-41e3-8bfa-6ce4bedf92e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.595358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc2d70-8c91-41e3-8bfa-6ce4bedf92e9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.606046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 807/1598] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.606046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84151838-de37-444b-82e9-1695838a03a6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.611258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 792/1599] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.611258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc2d70-8c91-41e3-8bfa-6ce4bedf92e9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.615517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84151838-de37-444b-82e9-1695838a03a6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.622488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9b721-e898-4da5-a74a-f16656b696fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.623098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9b721-e898-4da5-a74a-f16656b696fd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.623471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9b721-e898-4da5-a74a-f16656b696fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.623749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9b721-e898-4da5-a74a-f16656b696fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.624468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc2d70-8c91-41e3-8bfa-6ce4bedf92e9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.624600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9b721-e898-4da5-a74a-f16656b696fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.626159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9b721-e898-4da5-a74a-f16656b696fd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.630539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d5bb7-f1be-4d81-b9a9-df26a2c4cef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.631330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d5bb7-f1be-4d81-b9a9-df26a2c4cef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.631697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d5bb7-f1be-4d81-b9a9-df26a2c4cef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.632082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d5bb7-f1be-4d81-b9a9-df26a2c4cef3 admin admin] PATH_INFO: `/v3/projects/cbf14c554d854af5ba8cd910c82d42e0/users/4aaa94a183fb49b79e6383ea783ab76d/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.644667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 808/1600] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.645549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9b721-e898-4da5-a74a-f16656b696fd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.660150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d5bb7-f1be-4d81-b9a9-df26a2c4cef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf14c554d854af5ba8cd910c82d42e0, user_id=4aaa94a183fb49b79e6383ea783ab76d, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.663705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d5bb7-f1be-4d81-b9a9-df26a2c4cef3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.668608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9b721-e898-4da5-a74a-f16656b696fd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.679825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875afee-c00d-4e70-9419-4f19b58af01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.681296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875afee-c00d-4e70-9419-4f19b58af01b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.682119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875afee-c00d-4e70-9419-4f19b58af01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.683081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875afee-c00d-4e70-9419-4f19b58af01b admin admin] PATH_INFO: `/v3/projects/ea214b3e591a44928fd7af5f51095405/users/58fa0eb8e5ad41038d447c54bad91a6d/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.688683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 793/1601] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:57 2022] PUT /identity/v3/projects/cbf14c554d854af5ba8cd910c82d42e0/users/4aaa94a183fb49b79e6383ea783ab76d/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:57.690173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d5bb7-f1be-4d81-b9a9-df26a2c4cef3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.702678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:57.713829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875afee-c00d-4e70-9419-4f19b58af01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea214b3e591a44928fd7af5f51095405, user_id=58fa0eb8e5ad41038d447c54bad91a6d, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.717542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875afee-c00d-4e70-9419-4f19b58af01b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.742359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 809/1602] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:57 2022] PUT /identity/v3/projects/ea214b3e591a44928fd7af5f51095405/users/58fa0eb8e5ad41038d447c54bad91a6d/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:57.744268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875afee-c00d-4e70-9419-4f19b58af01b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.775015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:57.780071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d5bb7-f1be-4d81-b9a9-df26a2c4cef3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.858710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696ad66-4a4f-401f-a10d-991e20a68645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.859858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ad66-4a4f-401f-a10d-991e20a68645 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.860456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ad66-4a4f-401f-a10d-991e20a68645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.860944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696ad66-4a4f-401f-a10d-991e20a68645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.862221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696ad66-4a4f-401f-a10d-991e20a68645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.865473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696ad66-4a4f-401f-a10d-991e20a68645 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.872144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875afee-c00d-4e70-9419-4f19b58af01b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.877109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8303b7-af63-4258-adc4-599c8eedf859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.877799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8303b7-af63-4258-adc4-599c8eedf859 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.878159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8303b7-af63-4258-adc4-599c8eedf859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.878500 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8303b7-af63-4258-adc4-599c8eedf859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.879220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8303b7-af63-4258-adc4-599c8eedf859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.880985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8303b7-af63-4258-adc4-599c8eedf859 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.888705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 794/1603] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.889999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696ad66-4a4f-401f-a10d-991e20a68645 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.898693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 810/1604] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:57.898693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8303b7-af63-4258-adc4-599c8eedf859 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.909220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8303b7-af63-4258-adc4-599c8eedf859 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.909764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696ad66-4a4f-401f-a10d-991e20a68645 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:57.914237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cf47e-c4b3-470d-8d5b-04e9af6fb557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.915291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cf47e-c4b3-470d-8d5b-04e9af6fb557 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.915673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cf47e-c4b3-470d-8d5b-04e9af6fb557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.916055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cf47e-c4b3-470d-8d5b-04e9af6fb557 admin admin] PATH_INFO: `/v3/projects/dcca6c7d30114cd0936eb30381efad7d/users/218f3dd17e6549fc8796eb30e4b48e24/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.918609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5914c0-d3ee-4c05-aca2-77df726bd4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:57.920498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5914c0-d3ee-4c05-aca2-77df726bd4db admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.920498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5914c0-d3ee-4c05-aca2-77df726bd4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.920737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5914c0-d3ee-4c05-aca2-77df726bd4db admin admin] PATH_INFO: `/v3/projects/cbf73a31fbca41fd91cc218e11ccd2bb/users/b23f7d87039141e4ae29348d2fff2d36/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:57.937349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cf47e-c4b3-470d-8d5b-04e9af6fb557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcca6c7d30114cd0936eb30381efad7d, user_id=218f3dd17e6549fc8796eb30e4b48e24, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.940493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cf47e-c4b3-470d-8d5b-04e9af6fb557 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.946878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5914c0-d3ee-4c05-aca2-77df726bd4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf73a31fbca41fd91cc218e11ccd2bb, user_id=b23f7d87039141e4ae29348d2fff2d36, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:57.950820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5914c0-d3ee-4c05-aca2-77df726bd4db admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:57.966548 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 795/1605] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:57 2022] PUT /identity/v3/projects/cbf73a31fbca41fd91cc218e11ccd2bb/users/b23f7d87039141e4ae29348d2fff2d36/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:57.966783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 811/1606] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:57 2022] PUT /identity/v3/projects/dcca6c7d30114cd0936eb30381efad7d/users/218f3dd17e6549fc8796eb30e4b48e24/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:57.967099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5914c0-d3ee-4c05-aca2-77df726bd4db admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:57.971940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b6b62-7b8d-4bbd-afc7-875818089609 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:57.971940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b6b62-7b8d-4bbd-afc7-875818089609 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:57.971940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b6b62-7b8d-4bbd-afc7-875818089609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.009837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5f5b6b62-7b8d-4bbd-afc7-875818089609 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:58.012121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5f5b6b62-7b8d-4bbd-afc7-875818089609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:58.067183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5914c0-d3ee-4c05-aca2-77df726bd4db admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.074569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51d58a-be85-4326-96f3-b227c5b4ee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:58.075382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51d58a-be85-4326-96f3-b227c5b4ee51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.077039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51d58a-be85-4326-96f3-b227c5b4ee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.078607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51d58a-be85-4326-96f3-b227c5b4ee51 admin admin] PATH_INFO: `/v3/projects/c323083ac58246eb9339b61d7eb4bb80/users/5eb019fe2e374a3aa07f67aaad22ee05/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.083269 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 812/1607] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:58.087587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287aa3a-4703-4f07-91c6-5d65bd2418d3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.087874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287aa3a-4703-4f07-91c6-5d65bd2418d3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.088144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287aa3a-4703-4f07-91c6-5d65bd2418d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.110470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51d58a-be85-4326-96f3-b227c5b4ee51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c323083ac58246eb9339b61d7eb4bb80, user_id=5eb019fe2e374a3aa07f67aaad22ee05, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:58.113044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51d58a-be85-4326-96f3-b227c5b4ee51 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:58.122804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8287aa3a-4703-4f07-91c6-5d65bd2418d3 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:58.124609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8287aa3a-4703-4f07-91c6-5d65bd2418d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:58.138937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 796/1608] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:57 2022] PUT /identity/v3/projects/c323083ac58246eb9339b61d7eb4bb80/users/5eb019fe2e374a3aa07f67aaad22ee05/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:58.141138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51d58a-be85-4326-96f3-b227c5b4ee51 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:58.149884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:58.230602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 813/1609] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:58.231536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287aa3a-4703-4f07-91c6-5d65bd2418d3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:58.258583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287aa3a-4703-4f07-91c6-5d65bd2418d3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.266394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb3900-b60e-4f80-913d-740555da0e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:58.267134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3900-b60e-4f80-913d-740555da0e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.267420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3900-b60e-4f80-913d-740555da0e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.267692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3900-b60e-4f80-913d-740555da0e21 admin admin] PATH_INFO: `/v3/projects/9c882a78908b4497ab1a7d7d12d80a64/users/589414d8b665438497e7d9e00e6cfe75/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.280647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51d58a-be85-4326-96f3-b227c5b4ee51 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.297933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bebdf-16df-4aaa-82a4-658e3930e2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:58.298616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bebdf-16df-4aaa-82a4-658e3930e2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.298916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bebdf-16df-4aaa-82a4-658e3930e2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.299945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bebdf-16df-4aaa-82a4-658e3930e2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.299945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bebdf-16df-4aaa-82a4-658e3930e2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:58.301672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bebdf-16df-4aaa-82a4-658e3930e2ea admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:58.306473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb3900-b60e-4f80-913d-740555da0e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c882a78908b4497ab1a7d7d12d80a64, user_id=589414d8b665438497e7d9e00e6cfe75, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:58.313784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb3900-b60e-4f80-913d-740555da0e21 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:58.317877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 797/1610] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:58.317877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fa688-706d-4125-b65c-f6e9d8b022f4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.317877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fa688-706d-4125-b65c-f6e9d8b022f4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.317877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fa688-706d-4125-b65c-f6e9d8b022f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.344115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1f1fa688-706d-4125-b65c-f6e9d8b022f4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:58.345531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 814/1611] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:58 2022] PUT /identity/v3/projects/9c882a78908b4497ab1a7d7d12d80a64/users/589414d8b665438497e7d9e00e6cfe75/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:58.347699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8950dac-2f3c-44ec-880c-5d8ed7cddf61 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.349128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8950dac-2f3c-44ec-880c-5d8ed7cddf61 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.349420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8950dac-2f3c-44ec-880c-5d8ed7cddf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.349556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1f1fa688-706d-4125-b65c-f6e9d8b022f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:58.376548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:58.416004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a8950dac-2f3c-44ec-880c-5d8ed7cddf61 None None] MFA Rules not processed for user `b23f7d87039141e4ae29348d2fff2d36`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:58.418180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a8950dac-2f3c-44ec-880c-5d8ed7cddf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:58.475454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 798/1612] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:58.478691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fa688-706d-4125-b65c-f6e9d8b022f4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:58.556497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 815/1613] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:58.558408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8950dac-2f3c-44ec-880c-5d8ed7cddf61 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:58.585731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8950dac-2f3c-44ec-880c-5d8ed7cddf61 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.597482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039a19-fe0f-4a70-a3c8-a0d169286f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:58.598035 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039a19-fe0f-4a70-a3c8-a0d169286f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.598506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039a19-fe0f-4a70-a3c8-a0d169286f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.598921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039a19-fe0f-4a70-a3c8-a0d169286f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.599576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039a19-fe0f-4a70-a3c8-a0d169286f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:58.601385 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039a19-fe0f-4a70-a3c8-a0d169286f19 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:58.618516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fa688-706d-4125-b65c-f6e9d8b022f4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.629331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e366e19-4c04-4007-be9e-48839f78e12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:58.630293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e366e19-4c04-4007-be9e-48839f78e12b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.631824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 816/1614] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:22:58.632249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e366e19-4c04-4007-be9e-48839f78e12b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.633051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039a19-fe0f-4a70-a3c8-a0d169286f19 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:58.633689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e366e19-4c04-4007-be9e-48839f78e12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.648834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e366e19-4c04-4007-be9e-48839f78e12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:58.650515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e366e19-4c04-4007-be9e-48839f78e12b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:58.652631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039a19-fe0f-4a70-a3c8-a0d169286f19 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.657657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccef44a-5305-4d06-8303-e51670648333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:58.658277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccef44a-5305-4d06-8303-e51670648333 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.658554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccef44a-5305-4d06-8303-e51670648333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.658826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccef44a-5305-4d06-8303-e51670648333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.666294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 799/1615] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:22:58.673658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccef44a-5305-4d06-8303-e51670648333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:58.679514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccef44a-5305-4d06-8303-e51670648333 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:58.681260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e366e19-4c04-4007-be9e-48839f78e12b service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:58.694115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e366e19-4c04-4007-be9e-48839f78e12b service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.700667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5363-5af1-41ad-a4c4-235a4d729054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:58.701535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5363-5af1-41ad-a4c4-235a4d729054 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.701966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5363-5af1-41ad-a4c4-235a4d729054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.703142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e5363-5af1-41ad-a4c4-235a4d729054 admin admin] PATH_INFO: `/v3/projects/cae48cdfa9584978a977d9d6de7d0f10/users/d1b2541c13ac4984a55694899e2b73ec/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.720504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 817/1616] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:22:58.724012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01be663-136d-424c-8386-782177a6d810 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.726136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01be663-136d-424c-8386-782177a6d810 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.726423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01be663-136d-424c-8386-782177a6d810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.733549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5363-5af1-41ad-a4c4-235a4d729054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae48cdfa9584978a977d9d6de7d0f10, user_id=d1b2541c13ac4984a55694899e2b73ec, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:58.740796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e5363-5af1-41ad-a4c4-235a4d729054 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:58.768012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 800/1617] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:58 2022] PUT /identity/v3/projects/cae48cdfa9584978a977d9d6de7d0f10/users/d1b2541c13ac4984a55694899e2b73ec/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:58.769598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d01be663-136d-424c-8386-782177a6d810 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:58.770865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d01be663-136d-424c-8386-782177a6d810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:58.773930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5363-5af1-41ad-a4c4-235a4d729054 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:58.815757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:58.878203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 818/1618] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:58.882348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e5363-5af1-41ad-a4c4-235a4d729054 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.882690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5261d-605d-4906-8f2a-1916215e73d7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.883091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5261d-605d-4906-8f2a-1916215e73d7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.883423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5261d-605d-4906-8f2a-1916215e73d7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.885492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 819/1619] 10.222.0.75 () {54 vars in 962 bytes} [Fri Aug 19 15:22:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:22:58.887135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5261d-605d-4906-8f2a-1916215e73d7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:58.892683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9e4f0-a42d-4835-b128-34d71f9ca081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:58.893797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9e4f0-a42d-4835-b128-34d71f9ca081 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.893797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9e4f0-a42d-4835-b128-34d71f9ca081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.894030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9e4f0-a42d-4835-b128-34d71f9ca081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.903781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5261d-605d-4906-8f2a-1916215e73d7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.912412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea778bd-df6c-4e60-818d-41a55eaa2c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:58.913409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea778bd-df6c-4e60-818d-41a55eaa2c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.914027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea778bd-df6c-4e60-818d-41a55eaa2c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.914713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea778bd-df6c-4e60-818d-41a55eaa2c0b admin admin] PATH_INFO: `/v3/projects/c323083ac58246eb9339b61d7eb4bb80/users/5eb019fe2e374a3aa07f67aaad22ee05/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.919451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9e4f0-a42d-4835-b128-34d71f9ca081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:58.920170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea778bd-df6c-4e60-818d-41a55eaa2c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c323083ac58246eb9339b61d7eb4bb80, user_id=5eb019fe2e374a3aa07f67aaad22ee05, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:58.921575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9e4f0-a42d-4835-b128-34d71f9ca081 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:58.924772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea778bd-df6c-4e60-818d-41a55eaa2c0b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:58.940122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 820/1620] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:22:58 2022] PUT /identity/v3/projects/c323083ac58246eb9339b61d7eb4bb80/users/5eb019fe2e374a3aa07f67aaad22ee05/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:22:58.942293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f621f-2348-4447-8e10-573e8e32c2c2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:58.942737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f621f-2348-4447-8e10-573e8e32c2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:58.943316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f621f-2348-4447-8e10-573e8e32c2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:58.964879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 801/1621] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:22:58.965926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9e4f0-a42d-4835-b128-34d71f9ca081 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:58.986430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9e4f0-a42d-4835-b128-34d71f9ca081 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:58.987669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-257f621f-2348-4447-8e10-573e8e32c2c2 None None] MFA Rules not processed for user `d1b2541c13ac4984a55694899e2b73ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:58.989141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-257f621f-2348-4447-8e10-573e8e32c2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:59.050410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:59.081958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9941b8-7c25-4173-a295-5ecc0393f581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:59.083808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9941b8-7c25-4173-a295-5ecc0393f581 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:59.084092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9941b8-7c25-4173-a295-5ecc0393f581 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:59.084663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9941b8-7c25-4173-a295-5ecc0393f581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:59.101484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 821/1622] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:59.106103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a94d-957a-457a-99ed-34cf9eb51903 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:59.106621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a94d-957a-457a-99ed-34cf9eb51903 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:59.106906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556a94d-957a-457a-99ed-34cf9eb51903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:59.201654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8556a94d-957a-457a-99ed-34cf9eb51903 None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:59.203088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8556a94d-957a-457a-99ed-34cf9eb51903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:59.233916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9941b8-7c25-4173-a295-5ecc0393f581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:59.237736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9941b8-7c25-4173-a295-5ecc0393f581 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:59.268190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 802/1623] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:22:59.270534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472ed1-87b5-4043-9b6a-0af4bb26c55a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:59.271616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472ed1-87b5-4043-9b6a-0af4bb26c55a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:59.272036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80472ed1-87b5-4043-9b6a-0af4bb26c55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:59.342076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-80472ed1-87b5-4043-9b6a-0af4bb26c55a None None] MFA Rules not processed for user `5eb019fe2e374a3aa07f67aaad22ee05`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:22:59.350647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-80472ed1-87b5-4043-9b6a-0af4bb26c55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:22:59.405400 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 822/1624] 10.222.0.75 () {58 vars in 1059 bytes} [Fri Aug 19 15:22:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:59.420654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556a94d-957a-457a-99ed-34cf9eb51903 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:59.505800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:22:59.523815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 803/1625] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:22:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:22:59.523815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80472ed1-87b5-4043-9b6a-0af4bb26c55a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:59.534773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556a94d-957a-457a-99ed-34cf9eb51903 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:59.545803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d19a5-82f3-4a9b-84bb-029b991e6517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:59.546544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d19a5-82f3-4a9b-84bb-029b991e6517 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:59.546921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d19a5-82f3-4a9b-84bb-029b991e6517 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:59.547139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80472ed1-87b5-4043-9b6a-0af4bb26c55a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:59.547457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d19a5-82f3-4a9b-84bb-029b991e6517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:59.556230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57726fd5-ba74-42a4-b928-2424599be30b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:59.557493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57726fd5-ba74-42a4-b928-2424599be30b service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:59.558886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57726fd5-ba74-42a4-b928-2424599be30b service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:59.559250 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57726fd5-ba74-42a4-b928-2424599be30b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:59.566202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d19a5-82f3-4a9b-84bb-029b991e6517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:59.568846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d19a5-82f3-4a9b-84bb-029b991e6517 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:59.664998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 823/1626] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:22:59.664998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d19a5-82f3-4a9b-84bb-029b991e6517 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:22:59.664998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d19a5-82f3-4a9b-84bb-029b991e6517 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:22:59.664998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06d973-ab66-474f-a6b2-e479fa3cf2ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:22:59.668026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06d973-ab66-474f-a6b2-e479fa3cf2ff service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:22:59.668026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06d973-ab66-474f-a6b2-e479fa3cf2ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:22:59.668026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06d973-ab66-474f-a6b2-e479fa3cf2ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:22:59.735115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57726fd5-ba74-42a4-b928-2424599be30b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:59.736798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57726fd5-ba74-42a4-b928-2424599be30b service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:59.743819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06d973-ab66-474f-a6b2-e479fa3cf2ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:22:59.746107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06d973-ab66-474f-a6b2-e479fa3cf2ff service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:22:59.761110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 824/1627] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:22:59.767609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 804/1628] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:00.379301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0998e5-0fab-40cf-9eb4-3b8d44a8765a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:00.379741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0998e5-0fab-40cf-9eb4-3b8d44a8765a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:00.380198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0998e5-0fab-40cf-9eb4-3b8d44a8765a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:00.382157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 825/1629] 10.222.0.75 () {54 vars in 962 bytes} [Fri Aug 19 15:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:23:00.400809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c71055-234e-4fb7-9207-386978043683 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:00.402095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c71055-234e-4fb7-9207-386978043683 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:00.402683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c71055-234e-4fb7-9207-386978043683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:00.445242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-46c71055-234e-4fb7-9207-386978043683 None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:00.446555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-46c71055-234e-4fb7-9207-386978043683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:00.478410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 805/1630] 10.222.0.75 () {58 vars in 1059 bytes} [Fri Aug 19 15:23:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:00.485531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52702cc-b9eb-4c26-930c-2c4b26f5d7c8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:00.485531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52702cc-b9eb-4c26-930c-2c4b26f5d7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:00.485874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52702cc-b9eb-4c26-930c-2c4b26f5d7c8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:00.487463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 826/1631] 10.222.0.75 () {54 vars in 912 bytes} [Fri Aug 19 15:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:23:00.494345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f07fac-72b0-49b9-b9ad-2a483257ea5b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:00.494679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f07fac-72b0-49b9-b9ad-2a483257ea5b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:00.496123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f07fac-72b0-49b9-b9ad-2a483257ea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:00.548458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-88f07fac-72b0-49b9-b9ad-2a483257ea5b None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:00.551592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-88f07fac-72b0-49b9-b9ad-2a483257ea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:00.575417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 806/1632] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:23:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:00.755947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52702cc-b9eb-4c26-930c-2c4b26f5d7c8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:00.766127 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:00.777245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52702cc-b9eb-4c26-930c-2c4b26f5d7c8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:00.785489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af34d2-1e5b-46bd-8a83-927b6dbae0e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:00.788777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af34d2-1e5b-46bd-8a83-927b6dbae0e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:00.789263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af34d2-1e5b-46bd-8a83-927b6dbae0e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:00.790942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af34d2-1e5b-46bd-8a83-927b6dbae0e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:00.819825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af34d2-1e5b-46bd-8a83-927b6dbae0e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:00.823027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af34d2-1e5b-46bd-8a83-927b6dbae0e7 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:00.858876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 827/1633] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:01.033426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088431b-f332-4f3b-8db7-d397fb9a5045 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:01.034247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088431b-f332-4f3b-8db7-d397fb9a5045 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:01.034247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088431b-f332-4f3b-8db7-d397fb9a5045 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:01.036042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 807/1634] 10.222.0.75 () {54 vars in 912 bytes} [Fri Aug 19 15:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:23:01.045553 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7dabd-9a88-49bc-87ee-714b3d288e17 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:01.045992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7dabd-9a88-49bc-87ee-714b3d288e17 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:01.046462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7dabd-9a88-49bc-87ee-714b3d288e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:01.084939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bae7dabd-9a88-49bc-87ee-714b3d288e17 None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:01.088155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bae7dabd-9a88-49bc-87ee-714b3d288e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:01.117285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 828/1635] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:23:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:01.131043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088431b-f332-4f3b-8db7-d397fb9a5045 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:01.147033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088431b-f332-4f3b-8db7-d397fb9a5045 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:01.153129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ad7bc-b560-4d99-a97f-9a01c68bcba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:01.154293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ad7bc-b560-4d99-a97f-9a01c68bcba3 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:01.155532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ad7bc-b560-4d99-a97f-9a01c68bcba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:01.156055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ad7bc-b560-4d99-a97f-9a01c68bcba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:01.173661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ad7bc-b560-4d99-a97f-9a01c68bcba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:01.175235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ad7bc-b560-4d99-a97f-9a01c68bcba3 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:01.192790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 808/1636] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:01.335396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7dabd-9a88-49bc-87ee-714b3d288e17 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:01.349423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7dabd-9a88-49bc-87ee-714b3d288e17 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:01.356557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:01.357429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:01.357832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:01.358849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] PATH_INFO: `/v3/users/b23f7d87039141e4ae29348d2fff2d36` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:01.359893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23f7d87039141e4ae29348d2fff2d36)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:01.360835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:23:01.360835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 19 15:23:01.360835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: Aug 19 15:23:01.363065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:01.460250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23f7d87039141e4ae29348d2fff2d36'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:01.461374 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23f7d87039141e4ae29348d2fff2d36'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:01.462383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23f7d87039141e4ae29348d2fff2d36'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:01.487266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23f7d87039141e4ae29348d2fff2d36'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:01.533564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23f7d87039141e4ae29348d2fff2d36'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:01.576994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d9f8a-d77a-40bb-9bc9-304c34df8204 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:01.577476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d9f8a-d77a-40bb-9bc9-304c34df8204 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:01.577943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d9f8a-d77a-40bb-9bc9-304c34df8204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:01.582358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 829/1637] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:01 2022] DELETE /identity/v3/users/b23f7d87039141e4ae29348d2fff2d36 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:01.606205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-714d9f8a-d77a-40bb-9bc9-304c34df8204 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:01.608209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-714d9f8a-d77a-40bb-9bc9-304c34df8204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:01.723929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 809/1638] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:01.763168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:01.766512 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:01.942259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75a698-db89-4fa8-805e-7854419b774b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:01.955175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd968b9-0a79-4d28-a47c-8e2e2b0f253a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:01.956852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd968b9-0a79-4d28-a47c-8e2e2b0f253a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:01.957400 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd968b9-0a79-4d28-a47c-8e2e2b0f253a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:01.959727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd968b9-0a79-4d28-a47c-8e2e2b0f253a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:02.001933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd968b9-0a79-4d28-a47c-8e2e2b0f253a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:02.003894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd968b9-0a79-4d28-a47c-8e2e2b0f253a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:02.038952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 830/1639] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:02.746565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536b237-816b-4046-8d20-dacf366bf229 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:02.747239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536b237-816b-4046-8d20-dacf366bf229 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:02.747239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536b237-816b-4046-8d20-dacf366bf229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:02.806499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6536b237-816b-4046-8d20-dacf366bf229 None None] MFA Rules not processed for user `589414d8b665438497e7d9e00e6cfe75`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:02.806499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6536b237-816b-4046-8d20-dacf366bf229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:02.908077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:02.914096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 810/1640] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:02.919533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd968b9-0a79-4d28-a47c-8e2e2b0f253a service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:02.921306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5cd968b9-0a79-4d28-a47c-8e2e2b0f253a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:02.942523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd968b9-0a79-4d28-a47c-8e2e2b0f253a service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:02.953186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24ca2f-2c20-4c9c-8f32-476157031db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:02.954409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24ca2f-2c20-4c9c-8f32-476157031db6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:02.955795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24ca2f-2c20-4c9c-8f32-476157031db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:02.956357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24ca2f-2c20-4c9c-8f32-476157031db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:02.957363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24ca2f-2c20-4c9c-8f32-476157031db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:02.963155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24ca2f-2c20-4c9c-8f32-476157031db6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:03.035114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 831/1641] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:02 2022] POST /identity/v3/projects => generated 403 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:03.040327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536b237-816b-4046-8d20-dacf366bf229 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:03.058618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536b237-816b-4046-8d20-dacf366bf229 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:03.157608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ce775-0bdc-47f0-a183-82fbba7b62c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:03.159310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ce775-0bdc-47f0-a183-82fbba7b62c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:03.159597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ce775-0bdc-47f0-a183-82fbba7b62c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:03.159873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ce775-0bdc-47f0-a183-82fbba7b62c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:03.162705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ce775-0bdc-47f0-a183-82fbba7b62c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:03.166190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ce775-0bdc-47f0-a183-82fbba7b62c1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:03.224249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 811/1642] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:03 2022] POST /identity/v3/users => generated 308 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:03.228676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24ca2f-2c20-4c9c-8f32-476157031db6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:03.241529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24ca2f-2c20-4c9c-8f32-476157031db6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:03.248970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608d36f-7e7c-43b1-b42b-7c3e33686bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:03.249850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608d36f-7e7c-43b1-b42b-7c3e33686bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:03.250264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608d36f-7e7c-43b1-b42b-7c3e33686bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:03.250694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608d36f-7e7c-43b1-b42b-7c3e33686bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:03.251516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608d36f-7e7c-43b1-b42b-7c3e33686bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:03.253728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608d36f-7e7c-43b1-b42b-7c3e33686bb1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:03.277648 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ce775-0bdc-47f0-a183-82fbba7b62c1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:03.288621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 832/1643] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:03.297424 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ce775-0bdc-47f0-a183-82fbba7b62c1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:03.307609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f19fe9-744f-4486-941d-c123e4ec7ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:03.311382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f19fe9-744f-4486-941d-c123e4ec7ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:03.312152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f19fe9-744f-4486-941d-c123e4ec7ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:03.312877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f19fe9-744f-4486-941d-c123e4ec7ac5 admin admin] PATH_INFO: `/v3/projects/82751a9a158e4b0196f46e5bb4c4b9be/users/87392d83a6ac4ee9860619ac02324f26/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:03.343199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f19fe9-744f-4486-941d-c123e4ec7ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82751a9a158e4b0196f46e5bb4c4b9be, user_id=87392d83a6ac4ee9860619ac02324f26, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:03.347121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f19fe9-744f-4486-941d-c123e4ec7ac5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:03.395975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608d36f-7e7c-43b1-b42b-7c3e33686bb1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:03.405679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 812/1644] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:03 2022] PUT /identity/v3/projects/82751a9a158e4b0196f46e5bb4c4b9be/users/87392d83a6ac4ee9860619ac02324f26/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:03.415611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608d36f-7e7c-43b1-b42b-7c3e33686bb1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:03.476763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:03.505110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f19fe9-744f-4486-941d-c123e4ec7ac5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:03.507853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24f19fe9-744f-4486-941d-c123e4ec7ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:03.587610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736a805-81f7-4e24-bc82-91e631678529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:03.588458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736a805-81f7-4e24-bc82-91e631678529 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:03.589535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736a805-81f7-4e24-bc82-91e631678529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:03.590651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736a805-81f7-4e24-bc82-91e631678529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:03.592037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736a805-81f7-4e24-bc82-91e631678529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:03.595475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736a805-81f7-4e24-bc82-91e631678529 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:03.625158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 833/1645] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:03.630359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736a805-81f7-4e24-bc82-91e631678529 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:03.652426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736a805-81f7-4e24-bc82-91e631678529 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:03.679855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2f755-e7c5-4e0c-88f6-4df1b20fe7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:03.683153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2f755-e7c5-4e0c-88f6-4df1b20fe7fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:03.683586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2f755-e7c5-4e0c-88f6-4df1b20fe7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:03.685312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2f755-e7c5-4e0c-88f6-4df1b20fe7fe admin admin] PATH_INFO: `/v3/projects/82751a9a158e4b0196f46e5bb4c4b9be/users/87392d83a6ac4ee9860619ac02324f26/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:03.693221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2f755-e7c5-4e0c-88f6-4df1b20fe7fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=82751a9a158e4b0196f46e5bb4c4b9be, user_id=87392d83a6ac4ee9860619ac02324f26, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:03.716801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2f755-e7c5-4e0c-88f6-4df1b20fe7fe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:03.772942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 834/1646] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:03 2022] PUT /identity/v3/projects/82751a9a158e4b0196f46e5bb4c4b9be/users/87392d83a6ac4ee9860619ac02324f26/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:03.781374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f19fe9-744f-4486-941d-c123e4ec7ac5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:03.987005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:03.989074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:03.990636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:03.990928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] PATH_INFO: `/v3/projects/cbf73a31fbca41fd91cc218e11ccd2bb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:03.993433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf73a31fbca41fd91cc218e11ccd2bb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:03.999264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 15:23:03.999264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Aug 19 15:23:03.999264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: Aug 19 15:23:04.002953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:04.043722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:04.129936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf73a31fbca41fd91cc218e11ccd2bb', 'request_id': 'req-bc16f184-aac5-4791-9545-bc04aab3ce42'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:04.154586 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf73a31fbca41fd91cc218e11ccd2bb', 'request_id': 'req-bc16f184-aac5-4791-9545-bc04aab3ce42'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:04.183326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf73a31fbca41fd91cc218e11ccd2bb', 'request_id': 'req-bc16f184-aac5-4791-9545-bc04aab3ce42'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:04.237386 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] The token cache is being invalidate because project cbf73a31fbca41fd91cc218e11ccd2bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:04.237720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:04.252217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc16f184-aac5-4791-9545-bc04aab3ce42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:04.263344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 813/1647] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:03 2022] DELETE /identity/v3/projects/cbf73a31fbca41fd91cc218e11ccd2bb => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:04.265597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699e434-5fde-4ca8-8cd4-bc6e1c5c17a9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:04.266344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699e434-5fde-4ca8-8cd4-bc6e1c5c17a9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:04.266344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699e434-5fde-4ca8-8cd4-bc6e1c5c17a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:04.313250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e699e434-5fde-4ca8-8cd4-bc6e1c5c17a9 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:04.315658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e699e434-5fde-4ca8-8cd4-bc6e1c5c17a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:04.474530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 835/1648] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:04.481233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26219a-3474-42b5-8b3d-6692ca08db99 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:04.481734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26219a-3474-42b5-8b3d-6692ca08db99 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:04.482167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26219a-3474-42b5-8b3d-6692ca08db99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:04.559396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2a26219a-3474-42b5-8b3d-6692ca08db99 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:04.564071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2a26219a-3474-42b5-8b3d-6692ca08db99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:04.592529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 814/1649] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:04.596758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699e434-5fde-4ca8-8cd4-bc6e1c5c17a9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:04.621484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699e434-5fde-4ca8-8cd4-bc6e1c5c17a9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:04.645295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b6f19-edb7-4464-96e9-70eb15816922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:04.645941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b6f19-edb7-4464-96e9-70eb15816922 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:04.646222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b6f19-edb7-4464-96e9-70eb15816922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:04.646498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b6f19-edb7-4464-96e9-70eb15816922 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:04.647164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b6f19-edb7-4464-96e9-70eb15816922 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:04.647471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b6f19-edb7-4464-96e9-70eb15816922 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:04.653313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b6f19-edb7-4464-96e9-70eb15816922 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:04.699894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 836/1650] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:04.701164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26219a-3474-42b5-8b3d-6692ca08db99 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:04.717706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26219a-3474-42b5-8b3d-6692ca08db99 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:04.731326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b27fae-1a73-46df-abc5-c86fd8c00982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:04.731556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b27fae-1a73-46df-abc5-c86fd8c00982 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:04.731953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b27fae-1a73-46df-abc5-c86fd8c00982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:04.732123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b27fae-1a73-46df-abc5-c86fd8c00982 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:04.732912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b27fae-1a73-46df-abc5-c86fd8c00982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:04.746399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b27fae-1a73-46df-abc5-c86fd8c00982 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:04.843895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 815/1651] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:04 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:04.847843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b6f19-edb7-4464-96e9-70eb15816922 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:04.867154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b6f19-edb7-4464-96e9-70eb15816922 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:04.996031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e1dde-44c1-41a6-afb8-2e5fe3409e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:04.996721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e1dde-44c1-41a6-afb8-2e5fe3409e30 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:04.998058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e1dde-44c1-41a6-afb8-2e5fe3409e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:04.998058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e1dde-44c1-41a6-afb8-2e5fe3409e30 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:04.998058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e1dde-44c1-41a6-afb8-2e5fe3409e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:04.999983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e1dde-44c1-41a6-afb8-2e5fe3409e30 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:05.044319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 837/1652] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:04 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:05.048668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b27fae-1a73-46df-abc5-c86fd8c00982 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:05.063724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b27fae-1a73-46df-abc5-c86fd8c00982 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:05.076055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2194c0-351f-4958-8a63-0e8953fe94f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:05.076713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2194c0-351f-4958-8a63-0e8953fe94f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:05.077005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2194c0-351f-4958-8a63-0e8953fe94f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:05.077280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2194c0-351f-4958-8a63-0e8953fe94f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:05.078013 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2194c0-351f-4958-8a63-0e8953fe94f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:05.079661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2194c0-351f-4958-8a63-0e8953fe94f1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:05.103323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 816/1653] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:05.107775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e1dde-44c1-41a6-afb8-2e5fe3409e30 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:05.121673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e1dde-44c1-41a6-afb8-2e5fe3409e30 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:05.133451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f8d59-e33e-4531-81ec-9fec233238c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:05.134090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f8d59-e33e-4531-81ec-9fec233238c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:05.134369 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f8d59-e33e-4531-81ec-9fec233238c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:05.134642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f8d59-e33e-4531-81ec-9fec233238c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:05.135320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f8d59-e33e-4531-81ec-9fec233238c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:05.137160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f8d59-e33e-4531-81ec-9fec233238c8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:05.175488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 838/1654] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:05.180034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2194c0-351f-4958-8a63-0e8953fe94f1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:05.196542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2194c0-351f-4958-8a63-0e8953fe94f1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:05.222445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691c552-cc85-4557-a9bd-18da99c8def8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:05.223310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691c552-cc85-4557-a9bd-18da99c8def8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:05.223597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691c552-cc85-4557-a9bd-18da99c8def8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:05.224153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691c552-cc85-4557-a9bd-18da99c8def8 admin admin] PATH_INFO: `/v3/projects/6e638a932fa7446d906dec2b99f87bf0/users/9b6bda3d5b384c35aefe97cd8629dda4/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:05.265297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691c552-cc85-4557-a9bd-18da99c8def8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e638a932fa7446d906dec2b99f87bf0, user_id=9b6bda3d5b384c35aefe97cd8629dda4, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:05.268040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f691c552-cc85-4557-a9bd-18da99c8def8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:05.297797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 817/1655] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:05 2022] PUT /identity/v3/projects/6e638a932fa7446d906dec2b99f87bf0/users/9b6bda3d5b384c35aefe97cd8629dda4/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:05.309346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05fa9c-c8d4-4e40-9d8e-6ddbc5fd341a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:05.309936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05fa9c-c8d4-4e40-9d8e-6ddbc5fd341a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:05.309936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05fa9c-c8d4-4e40-9d8e-6ddbc5fd341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:05.369182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1b05fa9c-c8d4-4e40-9d8e-6ddbc5fd341a None None] MFA Rules not processed for user `9b6bda3d5b384c35aefe97cd8629dda4`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:05.371025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1b05fa9c-c8d4-4e40-9d8e-6ddbc5fd341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:05.388079 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:05.561453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 839/1656] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:05.567042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691c552-cc85-4557-a9bd-18da99c8def8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:05.804262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691c552-cc85-4557-a9bd-18da99c8def8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:05.815189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a60f-8472-463f-a1e0-b177a411e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:05.815832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a60f-8472-463f-a1e0-b177a411e55b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:05.816163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a60f-8472-463f-a1e0-b177a411e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:05.816573 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a60f-8472-463f-a1e0-b177a411e55b admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:05.818327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873a60f-8472-463f-a1e0-b177a411e55b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:05.829488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873a60f-8472-463f-a1e0-b177a411e55b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:05.894539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 818/1657] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:05 2022] POST /identity/v3/projects => generated 363 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:05.898321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05fa9c-c8d4-4e40-9d8e-6ddbc5fd341a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:05.915141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05fa9c-c8d4-4e40-9d8e-6ddbc5fd341a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:06.042465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2191d5-fb4d-4e79-b369-6e6ae3379e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:06.046686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2191d5-fb4d-4e79-b369-6e6ae3379e88 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:06.049349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2191d5-fb4d-4e79-b369-6e6ae3379e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:06.050026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2191d5-fb4d-4e79-b369-6e6ae3379e88 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:06.050938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2191d5-fb4d-4e79-b369-6e6ae3379e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:06.055468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2191d5-fb4d-4e79-b369-6e6ae3379e88 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:06.127521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 840/1658] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:05 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:06.131672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a60f-8472-463f-a1e0-b177a411e55b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:06.150082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a60f-8472-463f-a1e0-b177a411e55b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:06.159250 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13624cce-59ff-4c60-bc8f-dc235e0be16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:06.161796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13624cce-59ff-4c60-bc8f-dc235e0be16c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:06.162189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13624cce-59ff-4c60-bc8f-dc235e0be16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:06.162711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13624cce-59ff-4c60-bc8f-dc235e0be16c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:06.163904 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13624cce-59ff-4c60-bc8f-dc235e0be16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:06.192663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13624cce-59ff-4c60-bc8f-dc235e0be16c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:06.213306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 819/1659] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:06.217886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2191d5-fb4d-4e79-b369-6e6ae3379e88 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:06.234503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2191d5-fb4d-4e79-b369-6e6ae3379e88 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:06.239892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1eddd-3536-4d5f-be87-dc7243729b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:06.240733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1eddd-3536-4d5f-be87-dc7243729b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:06.241118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1eddd-3536-4d5f-be87-dc7243729b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:06.241761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1eddd-3536-4d5f-be87-dc7243729b92 admin admin] PATH_INFO: `/v3/projects/19b08e383f9a45e6b69ef6fd5dd4317d/users/9fe2b4556d63465ab1ea00a85b7c605a/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:06.267228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1eddd-3536-4d5f-be87-dc7243729b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b08e383f9a45e6b69ef6fd5dd4317d, user_id=9fe2b4556d63465ab1ea00a85b7c605a, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:06.270532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1eddd-3536-4d5f-be87-dc7243729b92 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:06.326033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 841/1660] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:06 2022] PUT /identity/v3/projects/19b08e383f9a45e6b69ef6fd5dd4317d/users/9fe2b4556d63465ab1ea00a85b7c605a/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:06.331104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13624cce-59ff-4c60-bc8f-dc235e0be16c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:06.357541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13624cce-59ff-4c60-bc8f-dc235e0be16c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:06.374222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:06.508775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae47261-2058-4827-9cf1-24468ea33ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:06.509663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae47261-2058-4827-9cf1-24468ea33ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:06.510085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae47261-2058-4827-9cf1-24468ea33ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:06.510497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae47261-2058-4827-9cf1-24468ea33ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:06.512186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae47261-2058-4827-9cf1-24468ea33ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:06.513985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae47261-2058-4827-9cf1-24468ea33ee2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:06.544829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 820/1661] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:06.551784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1eddd-3536-4d5f-be87-dc7243729b92 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:06.589945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1eddd-3536-4d5f-be87-dc7243729b92 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:06.632554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88e4df-9c43-4004-8831-6c5fc6eeff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:06.637473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88e4df-9c43-4004-8831-6c5fc6eeff30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:06.638058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88e4df-9c43-4004-8831-6c5fc6eeff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:06.638343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88e4df-9c43-4004-8831-6c5fc6eeff30 admin admin] PATH_INFO: `/v3/projects/19b08e383f9a45e6b69ef6fd5dd4317d/users/9fe2b4556d63465ab1ea00a85b7c605a/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:06.647348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88e4df-9c43-4004-8831-6c5fc6eeff30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b08e383f9a45e6b69ef6fd5dd4317d, user_id=9fe2b4556d63465ab1ea00a85b7c605a, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:06.663171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88e4df-9c43-4004-8831-6c5fc6eeff30 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:06.711427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 842/1662] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:06 2022] PUT /identity/v3/projects/19b08e383f9a45e6b69ef6fd5dd4317d/users/9fe2b4556d63465ab1ea00a85b7c605a/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:06.726957 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6917659-aa1d-4774-9b2d-dc8015e7748a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:06.726957 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6917659-aa1d-4774-9b2d-dc8015e7748a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:06.727446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6917659-aa1d-4774-9b2d-dc8015e7748a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:06.814952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a6917659-aa1d-4774-9b2d-dc8015e7748a None None] MFA Rules not processed for user `9fe2b4556d63465ab1ea00a85b7c605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:06.831262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a6917659-aa1d-4774-9b2d-dc8015e7748a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:06.985640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88e4df-9c43-4004-8831-6c5fc6eeff30 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:06.988356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc88e4df-9c43-4004-8831-6c5fc6eeff30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:07.189384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 821/1663] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:07.267513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6917659-aa1d-4774-9b2d-dc8015e7748a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:07.269809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a6917659-aa1d-4774-9b2d-dc8015e7748a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:07.395053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88e4df-9c43-4004-8831-6c5fc6eeff30 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:07.410899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16551123-752b-420a-9de4-e83e3feb5359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:07.411694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16551123-752b-420a-9de4-e83e3feb5359 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:07.412201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16551123-752b-420a-9de4-e83e3feb5359 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:07.412744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16551123-752b-420a-9de4-e83e3feb5359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:07.418336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-16551123-752b-420a-9de4-e83e3feb5359 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:07.449458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:07.559392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6917659-aa1d-4774-9b2d-dc8015e7748a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:07.571893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba74117-e832-4a7f-91b4-ceaaa41444fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:07.572566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba74117-e832-4a7f-91b4-ceaaa41444fe service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:07.572847 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba74117-e832-4a7f-91b4-ceaaa41444fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:07.573120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba74117-e832-4a7f-91b4-ceaaa41444fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:07.606285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba74117-e832-4a7f-91b4-ceaaa41444fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:07.608086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba74117-e832-4a7f-91b4-ceaaa41444fe service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:07.642988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 822/1664] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:07.673028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16551123-752b-420a-9de4-e83e3feb5359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:07.674920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16551123-752b-420a-9de4-e83e3feb5359 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:07.712209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 843/1665] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:07.893666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba74117-e832-4a7f-91b4-ceaaa41444fe service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:07.906826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba74117-e832-4a7f-91b4-ceaaa41444fe service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:07.911156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1882bdd-fd5d-48dd-9c45-433a5b5f3e0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:07.911799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1882bdd-fd5d-48dd-9c45-433a5b5f3e0b service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:07.912077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1882bdd-fd5d-48dd-9c45-433a5b5f3e0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:07.912346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1882bdd-fd5d-48dd-9c45-433a5b5f3e0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:08.002217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:08.094967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1882bdd-fd5d-48dd-9c45-433a5b5f3e0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:08.097129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1882bdd-fd5d-48dd-9c45-433a5b5f3e0b service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:08.142752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 823/1666] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:08.943831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50de16-9af9-4b9e-9b8c-af502648d829 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:08.944835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50de16-9af9-4b9e-9b8c-af502648d829 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:08.945276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50de16-9af9-4b9e-9b8c-af502648d829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:09.024735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9d50de16-9af9-4b9e-9b8c-af502648d829 None None] MFA Rules not processed for user `87392d83a6ac4ee9860619ac02324f26`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:09.028057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9d50de16-9af9-4b9e-9b8c-af502648d829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:09.243365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 844/1667] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:08 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:12.002909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665210a-3a57-4316-b2d2-da3a20a2fdd0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:12.004649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665210a-3a57-4316-b2d2-da3a20a2fdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:12.005295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665210a-3a57-4316-b2d2-da3a20a2fdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:12.071710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2665210a-3a57-4316-b2d2-da3a20a2fdd0 None None] MFA Rules not processed for user `dc2bbdceb12641e6a0595e68f7cefd63`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:12.073225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2665210a-3a57-4316-b2d2-da3a20a2fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:12.277020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 824/1668] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:12.292659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50de16-9af9-4b9e-9b8c-af502648d829 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:12.295527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9d50de16-9af9-4b9e-9b8c-af502648d829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:12.508362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50de16-9af9-4b9e-9b8c-af502648d829 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:12.540912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624102c-5852-41a8-b246-df3ef34946fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:12.543799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624102c-5852-41a8-b246-df3ef34946fa service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:12.544799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624102c-5852-41a8-b246-df3ef34946fa service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:12.545230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624102c-5852-41a8-b246-df3ef34946fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:12.567603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624102c-5852-41a8-b246-df3ef34946fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:12.571683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624102c-5852-41a8-b246-df3ef34946fa service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:12.600674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 845/1669] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:12.845181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62a6ba-2c40-4c0a-be69-0c18d96fb6c6 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:12.845464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62a6ba-2c40-4c0a-be69-0c18d96fb6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:12.845762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62a6ba-2c40-4c0a-be69-0c18d96fb6c6 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:12.847215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 825/1670] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:23:12.861719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030648c-1d1c-4577-86b3-774ecfd54cf5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:12.862038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030648c-1d1c-4577-86b3-774ecfd54cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:12.862277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030648c-1d1c-4577-86b3-774ecfd54cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:12.909286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0030648c-1d1c-4577-86b3-774ecfd54cf5 None None] MFA Rules not processed for user `bc2920aac4c046aca75473158ec204cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:12.912871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0030648c-1d1c-4577-86b3-774ecfd54cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:12.955337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:13.093333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 846/1671] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:13.125672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270b85a-d594-462b-a200-acbd3bcbfe9b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:13.126186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270b85a-d594-462b-a200-acbd3bcbfe9b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:13.126427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270b85a-d594-462b-a200-acbd3bcbfe9b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:13.129496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 826/1672] 10.222.0.75 () {54 vars in 961 bytes} [Fri Aug 19 15:23:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:23:13.135128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277813-a9df-4da8-b0eb-d66a0685dce8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:13.136181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277813-a9df-4da8-b0eb-d66a0685dce8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:13.136755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a277813-a9df-4da8-b0eb-d66a0685dce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:13.171541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1a277813-a9df-4da8-b0eb-d66a0685dce8 None None] MFA Rules not processed for user `bc2920aac4c046aca75473158ec204cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:13.173769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1a277813-a9df-4da8-b0eb-d66a0685dce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:13.192147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 847/1673] 10.222.0.75 () {58 vars in 1058 bytes} [Fri Aug 19 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:13.201456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270b85a-d594-462b-a200-acbd3bcbfe9b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:13.221062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270b85a-d594-462b-a200-acbd3bcbfe9b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:13.229259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959a4ce-4dc4-45db-9c5f-c73a61a09388 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:13.230341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959a4ce-4dc4-45db-9c5f-c73a61a09388 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:13.230775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959a4ce-4dc4-45db-9c5f-c73a61a09388 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:13.231155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959a4ce-4dc4-45db-9c5f-c73a61a09388 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:13.246747 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959a4ce-4dc4-45db-9c5f-c73a61a09388 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:13.248934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959a4ce-4dc4-45db-9c5f-c73a61a09388 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:13.288788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 827/1674] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:13.454643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a277813-a9df-4da8-b0eb-d66a0685dce8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:13.638640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a277813-a9df-4da8-b0eb-d66a0685dce8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:13.662447 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:13.663170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:13.663452 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:13.663725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] PATH_INFO: `/v3/users/9b6bda3d5b384c35aefe97cd8629dda4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:13.672360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b6bda3d5b384c35aefe97cd8629dda4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:13.679535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:13.821063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b6bda3d5b384c35aefe97cd8629dda4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:13.822635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b6bda3d5b384c35aefe97cd8629dda4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:13.824229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b6bda3d5b384c35aefe97cd8629dda4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:13.843664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b6bda3d5b384c35aefe97cd8629dda4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:13.883277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b6bda3d5b384c35aefe97cd8629dda4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:13.911064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 848/1675] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:13 2022] DELETE /identity/v3/users/9b6bda3d5b384c35aefe97cd8629dda4 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:13.915646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c8358-fdd9-4c1f-930c-14ec12c9aa98 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:13.915936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c8358-fdd9-4c1f-930c-14ec12c9aa98 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:13.916217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c8358-fdd9-4c1f-930c-14ec12c9aa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:13.970481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-177c8358-fdd9-4c1f-930c-14ec12c9aa98 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:13.977682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-177c8358-fdd9-4c1f-930c-14ec12c9aa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:14.165426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 828/1676] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:14.179507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:14.181120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:14.275078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:14.342638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004683a-19da-422e-85e3-7920c4d3893c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:14.353777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60782079-b10d-4f42-944f-4b19dfec845b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:14.354012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60782079-b10d-4f42-944f-4b19dfec845b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:14.354330 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60782079-b10d-4f42-944f-4b19dfec845b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:14.358454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60782079-b10d-4f42-944f-4b19dfec845b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:14.383791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60782079-b10d-4f42-944f-4b19dfec845b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:14.385427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60782079-b10d-4f42-944f-4b19dfec845b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:14.417399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 849/1677] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:15.268036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723871-2ff5-4812-a96b-8adcd99e8a32 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:15.269007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723871-2ff5-4812-a96b-8adcd99e8a32 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:15.269435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e723871-2ff5-4812-a96b-8adcd99e8a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:15.305316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f746f-0498-4a5c-83dc-1a0ea5b778e1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:15.305836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f746f-0498-4a5c-83dc-1a0ea5b778e1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:15.305836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f746f-0498-4a5c-83dc-1a0ea5b778e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:15.325981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5e723871-2ff5-4812-a96b-8adcd99e8a32 None None] MFA Rules not processed for user `58fa0eb8e5ad41038d447c54bad91a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:15.327234 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5e723871-2ff5-4812-a96b-8adcd99e8a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:15.373260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ab7f746f-0498-4a5c-83dc-1a0ea5b778e1 None None] MFA Rules not processed for user `4aaa94a183fb49b79e6383ea783ab76d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:15.376593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ab7f746f-0498-4a5c-83dc-1a0ea5b778e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:15.457913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:15.532776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 829/1678] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:15.534568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bd8d6-ef73-4f01-92b5-08d1850f03a3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:15.536857 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bd8d6-ef73-4f01-92b5-08d1850f03a3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:15.537284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96bd8d6-ef73-4f01-92b5-08d1850f03a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:15.580284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a96bd8d6-ef73-4f01-92b5-08d1850f03a3 None None] MFA Rules not processed for user `58fa0eb8e5ad41038d447c54bad91a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:15.582786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 850/1679] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:15.583982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a96bd8d6-ef73-4f01-92b5-08d1850f03a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:15.584505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f746f-0498-4a5c-83dc-1a0ea5b778e1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:15.587611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ab7f746f-0498-4a5c-83dc-1a0ea5b778e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:15.615908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f746f-0498-4a5c-83dc-1a0ea5b778e1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:15.627516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:15.627758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 830/1680] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:15.628270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:15.628652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:15.628932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] PATH_INFO: `/v3/projects/6e638a932fa7446d906dec2b99f87bf0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:15.629170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bd8d6-ef73-4f01-92b5-08d1850f03a3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:15.630683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a96bd8d6-ef73-4f01-92b5-08d1850f03a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:15.633138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e638a932fa7446d906dec2b99f87bf0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:15.635398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:15.715045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e638a932fa7446d906dec2b99f87bf0', 'request_id': 'req-807512ef-ee0f-45c5-9ebd-d9649851d849'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:15.726027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e638a932fa7446d906dec2b99f87bf0', 'request_id': 'req-807512ef-ee0f-45c5-9ebd-d9649851d849'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:15.745165 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e638a932fa7446d906dec2b99f87bf0', 'request_id': 'req-807512ef-ee0f-45c5-9ebd-d9649851d849'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:15.769026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] The token cache is being invalidate because project 6e638a932fa7446d906dec2b99f87bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:15.769349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:15.770057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:15.770982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96bd8d6-ef73-4f01-92b5-08d1850f03a3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:15.771994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 851/1681] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:15 2022] DELETE /identity/v3/projects/6e638a932fa7446d906dec2b99f87bf0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:15.772957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:15.777082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:15.876622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:15.881183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:15.881813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:15.882156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:15.882485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:15.893609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:15.993343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807512ef-ee0f-45c5-9ebd-d9649851d849 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:16.007530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:16.007851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:16.008137 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:16.008433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:16.016267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:16.021178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:16.023668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:16.037896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 831/1682] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:16.038826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:16.041596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:16.058191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:16.175296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edfc0c-3509-4f9f-9541-7ab91013469c service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:16.188259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:16.188955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:16.189240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:16.189514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] PATH_INFO: `/v3/users/9fe2b4556d63465ab1ea00a85b7c605a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:16.190724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe2b4556d63465ab1ea00a85b7c605a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:16.193021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:16.222913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:16.226006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:16.258731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 852/1683] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:16.271017 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:16.277045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:16.352724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe2b4556d63465ab1ea00a85b7c605a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.359511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4289f-2fb4-455b-bf28-b03886237363 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:16.376956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe2b4556d63465ab1ea00a85b7c605a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.377788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe2b4556d63465ab1ea00a85b7c605a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.383062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:16.384246 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:16.384557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:16.384930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] PATH_INFO: `/v3/users/d1b2541c13ac4984a55694899e2b73ec` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:16.390600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b2541c13ac4984a55694899e2b73ec)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:16.399591 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:16.445737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe2b4556d63465ab1ea00a85b7c605a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.478756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-430473ee-10de-4d05-9066-19695ac0e169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe2b4556d63465ab1ea00a85b7c605a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.486731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 832/1684] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:16 2022] DELETE /identity/v3/users/9fe2b4556d63465ab1ea00a85b7c605a => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:16.488087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d413e-4188-4f29-b5e3-8f444b775629 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:16.488378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d413e-4188-4f29-b5e3-8f444b775629 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:16.488683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d413e-4188-4f29-b5e3-8f444b775629 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:16.501029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 833/1685] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:23:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:23:16.503661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d413e-4188-4f29-b5e3-8f444b775629 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:16.513835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0c8d413e-4188-4f29-b5e3-8f444b775629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:16.604244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b2541c13ac4984a55694899e2b73ec'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.605379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b2541c13ac4984a55694899e2b73ec'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.606275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b2541c13ac4984a55694899e2b73ec'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.618598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b2541c13ac4984a55694899e2b73ec'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.658969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1509291b-46e1-4381-a283-0366916382c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b2541c13ac4984a55694899e2b73ec'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:16.693206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 853/1686] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:16 2022] DELETE /identity/v3/users/d1b2541c13ac4984a55694899e2b73ec => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:16.695030 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2b64-c0c7-4bb3-bd7e-acc9eea21e80 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:16.695321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2b64-c0c7-4bb3-bd7e-acc9eea21e80 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:16.697795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd2b64-c0c7-4bb3-bd7e-acc9eea21e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:16.722425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d413e-4188-4f29-b5e3-8f444b775629 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:16.734428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-32cd2b64-c0c7-4bb3-bd7e-acc9eea21e80 None None] MFA Rules not processed for user `bc2920aac4c046aca75473158ec204cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:16.736133 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-32cd2b64-c0c7-4bb3-bd7e-acc9eea21e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:16.813054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:16.838546 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:16.878234 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97865929-0410-4762-b934-fd5490399448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:16.879461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97865929-0410-4762-b934-fd5490399448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:16.879748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97865929-0410-4762-b934-fd5490399448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:16.880370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97865929-0410-4762-b934-fd5490399448 admin admin] PATH_INFO: `/v3/users/589414d8b665438497e7d9e00e6cfe75` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:16.884295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97865929-0410-4762-b934-fd5490399448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=589414d8b665438497e7d9e00e6cfe75)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:16.886811 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 854/1687] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:16.888850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97865929-0410-4762-b934-fd5490399448 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:16.891561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f1311-7ba5-4415-9bbd-10a8f34def1a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:16.891958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f1311-7ba5-4415-9bbd-10a8f34def1a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:16.893163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f1311-7ba5-4415-9bbd-10a8f34def1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:16.951391 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-882f1311-7ba5-4415-9bbd-10a8f34def1a None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:16.955208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-882f1311-7ba5-4415-9bbd-10a8f34def1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:16.992771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 855/1688] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:16.993853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f1311-7ba5-4415-9bbd-10a8f34def1a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:17.001288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-882f1311-7ba5-4415-9bbd-10a8f34def1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:17.042792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-97865929-0410-4762-b934-fd5490399448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589414d8b665438497e7d9e00e6cfe75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:17.051742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f1311-7ba5-4415-9bbd-10a8f34def1a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:17.059567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-97865929-0410-4762-b934-fd5490399448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589414d8b665438497e7d9e00e6cfe75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:17.061314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-97865929-0410-4762-b934-fd5490399448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589414d8b665438497e7d9e00e6cfe75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:17.065258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9275b-adda-4bbf-a8fe-2f62d7738448 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:17.068025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9275b-adda-4bbf-a8fe-2f62d7738448 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:17.068309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9275b-adda-4bbf-a8fe-2f62d7738448 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:17.068611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9275b-adda-4bbf-a8fe-2f62d7738448 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:17.118213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9275b-adda-4bbf-a8fe-2f62d7738448 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:17.123324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-97865929-0410-4762-b934-fd5490399448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589414d8b665438497e7d9e00e6cfe75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:17.124577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9275b-adda-4bbf-a8fe-2f62d7738448 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:17.150904 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-97865929-0410-4762-b934-fd5490399448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589414d8b665438497e7d9e00e6cfe75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:17.170459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 834/1689] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:16 2022] DELETE /identity/v3/users/589414d8b665438497e7d9e00e6cfe75 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:17.174361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 856/1690] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:17.316628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97865929-0410-4762-b934-fd5490399448 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:17.318490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-97865929-0410-4762-b934-fd5490399448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:17.478659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97865929-0410-4762-b934-fd5490399448 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:17.486402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdccc7-626f-4055-ad57-5e92d2d9670d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:17.487490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdccc7-626f-4055-ad57-5e92d2d9670d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:17.488638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdccc7-626f-4055-ad57-5e92d2d9670d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:17.489071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdccc7-626f-4055-ad57-5e92d2d9670d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:17.492219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6ffdccc7-626f-4055-ad57-5e92d2d9670d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:17.566732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33e4d2-14d3-4761-84b2-2070e9d74fa2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:17.567706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33e4d2-14d3-4761-84b2-2070e9d74fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:17.568638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33e4d2-14d3-4761-84b2-2070e9d74fa2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:17.571711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 857/1691] 10.222.0.75 () {54 vars in 912 bytes} [Fri Aug 19 15:23:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:23:17.586063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75871666-3826-4611-b156-cca2187d261a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:17.586668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75871666-3826-4611-b156-cca2187d261a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:17.587081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75871666-3826-4611-b156-cca2187d261a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:17.614027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdccc7-626f-4055-ad57-5e92d2d9670d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:17.616380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdccc7-626f-4055-ad57-5e92d2d9670d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:17.628993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 835/1692] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:17.664407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-75871666-3826-4611-b156-cca2187d261a None None] MFA Rules not processed for user `c8704aae85084d41bdedb5537ef96e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:17.667552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-75871666-3826-4611-b156-cca2187d261a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:17.685534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43f6c7-c5ee-425e-9f14-6f694c34a840 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:17.685534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43f6c7-c5ee-425e-9f14-6f694c34a840 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:17.685813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43f6c7-c5ee-425e-9f14-6f694c34a840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:17.701929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:17.744487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 858/1693] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:23:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:17.750683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fd43f6c7-c5ee-425e-9f14-6f694c34a840 None None] MFA Rules not processed for user `218f3dd17e6549fc8796eb30e4b48e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:17.752284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fd43f6c7-c5ee-425e-9f14-6f694c34a840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:17.773627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75871666-3826-4611-b156-cca2187d261a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:17.798901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75871666-3826-4611-b156-cca2187d261a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:17.806887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d74f73-9a5a-4031-aeaf-588eb75ad70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:17.808028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d74f73-9a5a-4031-aeaf-588eb75ad70c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:17.809018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d74f73-9a5a-4031-aeaf-588eb75ad70c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:17.809364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d74f73-9a5a-4031-aeaf-588eb75ad70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:17.832428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d74f73-9a5a-4031-aeaf-588eb75ad70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:17.836237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d74f73-9a5a-4031-aeaf-588eb75ad70c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:17.861871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 859/1694] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:17.917156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 836/1695] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:17.931275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1790a-27e7-473c-a67c-d1548a9dd706 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:17.931275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1790a-27e7-473c-a67c-d1548a9dd706 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:17.931275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1790a-27e7-473c-a67c-d1548a9dd706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:17.975977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a0d1790a-27e7-473c-a67c-d1548a9dd706 None None] MFA Rules not processed for user `218f3dd17e6549fc8796eb30e4b48e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:17.982172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a0d1790a-27e7-473c-a67c-d1548a9dd706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:17.998516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 860/1696] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:18.009623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43f6c7-c5ee-425e-9f14-6f694c34a840 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:18.015442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fd43f6c7-c5ee-425e-9f14-6f694c34a840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:18.367590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1790a-27e7-473c-a67c-d1548a9dd706 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:18.369214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a0d1790a-27e7-473c-a67c-d1548a9dd706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:18.378982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43f6c7-c5ee-425e-9f14-6f694c34a840 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:18.389331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8aafbf-7b29-49bf-8f2e-1e2057f2598a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:18.390329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8aafbf-7b29-49bf-8f2e-1e2057f2598a service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:18.390612 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8aafbf-7b29-49bf-8f2e-1e2057f2598a service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:18.390889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8aafbf-7b29-49bf-8f2e-1e2057f2598a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:18.393557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1790a-27e7-473c-a67c-d1548a9dd706 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:18.416998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:18.418341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:18.418626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:18.418903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] PATH_INFO: `/v3/projects/9c882a78908b4497ab1a7d7d12d80a64` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:18.421109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c882a78908b4497ab1a7d7d12d80a64)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:18.425363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8aafbf-7b29-49bf-8f2e-1e2057f2598a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:18.425520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:18.430804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8aafbf-7b29-49bf-8f2e-1e2057f2598a service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:18.480414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 837/1697] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:18.560891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c882a78908b4497ab1a7d7d12d80a64', 'request_id': 'req-a287d176-f9e1-458d-a4a0-0d65504821e0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:18.575976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c882a78908b4497ab1a7d7d12d80a64', 'request_id': 'req-a287d176-f9e1-458d-a4a0-0d65504821e0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:18.599359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c882a78908b4497ab1a7d7d12d80a64', 'request_id': 'req-a287d176-f9e1-458d-a4a0-0d65504821e0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:18.616150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8aafbf-7b29-49bf-8f2e-1e2057f2598a service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:18.617681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5f8aafbf-7b29-49bf-8f2e-1e2057f2598a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:18.631135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] The token cache is being invalidate because project 9c882a78908b4497ab1a7d7d12d80a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:18.631509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:18.636231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:18.643664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 861/1698] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:18 2022] DELETE /identity/v3/projects/9c882a78908b4497ab1a7d7d12d80a64 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:18.644871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:18.652606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8aafbf-7b29-49bf-8f2e-1e2057f2598a service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:18.654264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:18.899388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:18.900067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:18.900360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:18.900670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] PATH_INFO: `/v3/projects/19b08e383f9a45e6b69ef6fd5dd4317d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:18.906882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b08e383f9a45e6b69ef6fd5dd4317d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:18.908679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:18.927581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287d176-f9e1-458d-a4a0-0d65504821e0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:18.943562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:18.944288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:18.944596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:18.944878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] PATH_INFO: `/v3/users/87392d83a6ac4ee9860619ac02324f26` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:18.946407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87392d83a6ac4ee9860619ac02324f26)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:18.954844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:19.002954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:19.038021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:19.103307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b08e383f9a45e6b69ef6fd5dd4317d', 'request_id': 'req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.103522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87392d83a6ac4ee9860619ac02324f26'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.104097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87392d83a6ac4ee9860619ac02324f26'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.107491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87392d83a6ac4ee9860619ac02324f26'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.119902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b08e383f9a45e6b69ef6fd5dd4317d', 'request_id': 'req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.125591 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87392d83a6ac4ee9860619ac02324f26'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.137820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b08e383f9a45e6b69ef6fd5dd4317d', 'request_id': 'req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.168011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] The token cache is being invalidate because project 19b08e383f9a45e6b69ef6fd5dd4317d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:19.168350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.169298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b5ce3ac-b215-4a9d-be5e-3b89c0d84c45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.170358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad89810a-7ea4-4950-ac4b-f3f06efaf781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87392d83a6ac4ee9860619ac02324f26'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.173361 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 838/1699] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:18 2022] DELETE /identity/v3/projects/19b08e383f9a45e6b69ef6fd5dd4317d => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:19.184881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc155e-af91-4f52-a3f8-7026428d2240 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:19.185248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc155e-af91-4f52-a3f8-7026428d2240 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:19.185609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc155e-af91-4f52-a3f8-7026428d2240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:19.207677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 862/1700] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:18 2022] DELETE /identity/v3/users/87392d83a6ac4ee9860619ac02324f26 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:19.230992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-65cc155e-af91-4f52-a3f8-7026428d2240 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:19.233519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-65cc155e-af91-4f52-a3f8-7026428d2240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:19.414532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 839/1701] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:19.417492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dda9cf-ea25-498a-8e0a-442641583679 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:19.417919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dda9cf-ea25-498a-8e0a-442641583679 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:19.418451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dda9cf-ea25-498a-8e0a-442641583679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:19.457815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675bbb6-7642-4a21-bc83-c0b2659b122f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:19.458096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675bbb6-7642-4a21-bc83-c0b2659b122f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:19.458833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675bbb6-7642-4a21-bc83-c0b2659b122f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:19.460545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 840/1702] 10.222.0.75 () {54 vars in 912 bytes} [Fri Aug 19 15:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:23:19.475316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48911d54-9873-4b04-8ce1-8b1878d01f6b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:19.475801 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48911d54-9873-4b04-8ce1-8b1878d01f6b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:19.476500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48911d54-9873-4b04-8ce1-8b1878d01f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:19.503397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c5dda9cf-ea25-498a-8e0a-442641583679 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:19.510422 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c5dda9cf-ea25-498a-8e0a-442641583679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:19.547330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-48911d54-9873-4b04-8ce1-8b1878d01f6b None None] MFA Rules not processed for user `c8704aae85084d41bdedb5537ef96e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:19.549102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-48911d54-9873-4b04-8ce1-8b1878d01f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:19.556157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 863/1703] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:19.557243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dda9cf-ea25-498a-8e0a-442641583679 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:19.564854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c5dda9cf-ea25-498a-8e0a-442641583679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:19.609192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dda9cf-ea25-498a-8e0a-442641583679 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:19.659321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:19.660000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:19.660280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:19.660590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] PATH_INFO: `/v3/projects/cae48cdfa9584978a977d9d6de7d0f10` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:19.673426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae48cdfa9584978a977d9d6de7d0f10)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:19.675456 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:19.849483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 841/1704] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:23:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:19.849483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48911d54-9873-4b04-8ce1-8b1878d01f6b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:19.866715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48911d54-9873-4b04-8ce1-8b1878d01f6b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:19.884521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:19.885524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:19.886623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:19.887041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:19.887955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:19.888590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:19.890453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:19.916988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 842/1705] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:19.918699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:19.918993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae48cdfa9584978a977d9d6de7d0f10', 'request_id': 'req-7489dfcc-011a-4007-a748-d4da00797549'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.921629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:19.940208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae48cdfa9584978a977d9d6de7d0f10', 'request_id': 'req-7489dfcc-011a-4007-a748-d4da00797549'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:19.943866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5368f9-8680-4e45-a478-bf12eec3271a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:19.966018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae48cdfa9584978a977d9d6de7d0f10', 'request_id': 'req-7489dfcc-011a-4007-a748-d4da00797549'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:20.003844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] The token cache is being invalidate because project cae48cdfa9584978a977d9d6de7d0f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:20.004287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:20.006458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:20.013563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 864/1706] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:19 2022] DELETE /identity/v3/projects/cae48cdfa9584978a977d9d6de7d0f10 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:20.013563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:20.017184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.049539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:20.077017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:20.077908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:20.078187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:20.078462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] PATH_INFO: `/v3/projects/82751a9a158e4b0196f46e5bb4c4b9be` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:20.079838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82751a9a158e4b0196f46e5bb4c4b9be)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:20.082080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:20.106600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489dfcc-011a-4007-a748-d4da00797549 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:20.116341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:20.117036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:20.117323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:20.117598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:20.121119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.143648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:20.145238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:20.160559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 865/1707] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:20.161412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:20.161720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82751a9a158e4b0196f46e5bb4c4b9be', 'request_id': 'req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:20.165489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.180380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82751a9a158e4b0196f46e5bb4c4b9be', 'request_id': 'req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:20.194374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82751a9a158e4b0196f46e5bb4c4b9be', 'request_id': 'req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:20.203837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f6b0b-892b-4a86-9516-e4fd8c41e0a2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:20.212904 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] The token cache is being invalidate because project 82751a9a158e4b0196f46e5bb4c4b9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:20.213251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:20.214149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:20.216624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:20.218455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 843/1708] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:19 2022] DELETE /identity/v3/projects/82751a9a158e4b0196f46e5bb4c4b9be => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:20.219922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:20.223989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.343870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1c09d-bc3e-4610-b1c9-394aae34b2f7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:20.350500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f4040-fada-4caa-bad5-0955aaa44930 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:20.351135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f4040-fada-4caa-bad5-0955aaa44930 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:20.351415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f4040-fada-4caa-bad5-0955aaa44930 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:20.351690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f4040-fada-4caa-bad5-0955aaa44930 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:20.355364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a69f4040-fada-4caa-bad5-0955aaa44930 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.376126 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f4040-fada-4caa-bad5-0955aaa44930 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:20.377814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f4040-fada-4caa-bad5-0955aaa44930 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:20.415322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 844/1709] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:20.419297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670e552-6a49-41c7-8547-cb1198b3ef62 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:20.419453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670e552-6a49-41c7-8547-cb1198b3ef62 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:20.419758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670e552-6a49-41c7-8547-cb1198b3ef62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:20.420995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897c974-5f46-4118-8988-4ef8663885cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:20.422181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897c974-5f46-4118-8988-4ef8663885cf admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:20.422799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897c974-5f46-4118-8988-4ef8663885cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:20.423399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897c974-5f46-4118-8988-4ef8663885cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:20.425589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897c974-5f46-4118-8988-4ef8663885cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:20.436565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1897c974-5f46-4118-8988-4ef8663885cf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:20.470859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5670e552-6a49-41c7-8547-cb1198b3ef62 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:20.472372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5670e552-6a49-41c7-8547-cb1198b3ef62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.498561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 845/1710] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:20.498561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143c965-85f3-4703-9843-b8fe01694959 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:20.498561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143c965-85f3-4703-9843-b8fe01694959 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:20.500021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143c965-85f3-4703-9843-b8fe01694959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:20.538582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 866/1711] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:20 2022] POST /identity/v3/projects => generated 369 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:20.541278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24960a6-0fe8-459d-8ca8-5ea7a55555ab None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:20.549781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e143c965-85f3-4703-9843-b8fe01694959 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:20.551050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e143c965-85f3-4703-9843-b8fe01694959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.556204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24960a6-0fe8-459d-8ca8-5ea7a55555ab None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:20.557538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24960a6-0fe8-459d-8ca8-5ea7a55555ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:20.660145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d24960a6-0fe8-459d-8ca8-5ea7a55555ab None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:20.679455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d24960a6-0fe8-459d-8ca8-5ea7a55555ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.713755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 846/1712] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:20.718146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143c965-85f3-4703-9843-b8fe01694959 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:20.721450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e143c965-85f3-4703-9843-b8fe01694959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.738900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:20.748372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143c965-85f3-4703-9843-b8fe01694959 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:20.763532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13e1b2-0a26-49d8-ac8d-e99aeb1a566b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:20.764151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13e1b2-0a26-49d8-ac8d-e99aeb1a566b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:20.764810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13e1b2-0a26-49d8-ac8d-e99aeb1a566b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:20.764810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13e1b2-0a26-49d8-ac8d-e99aeb1a566b admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:20.791270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13e1b2-0a26-49d8-ac8d-e99aeb1a566b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:20.795211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13e1b2-0a26-49d8-ac8d-e99aeb1a566b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:20.896375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 847/1713] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:20 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:20.897992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0aba5-f44a-4cee-b94f-40c4a90b8a54 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:20.898278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0aba5-f44a-4cee-b94f-40c4a90b8a54 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:20.899153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0aba5-f44a-4cee-b94f-40c4a90b8a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:20.962370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d6b0aba5-f44a-4cee-b94f-40c4a90b8a54 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:20.967232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d6b0aba5-f44a-4cee-b94f-40c4a90b8a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:20.977895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 867/1714] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:20.979778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24960a6-0fe8-459d-8ca8-5ea7a55555ab None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:20.991604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 848/1715] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:20.992528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0aba5-f44a-4cee-b94f-40c4a90b8a54 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:21.018314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24960a6-0fe8-459d-8ca8-5ea7a55555ab None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:21.019756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0aba5-f44a-4cee-b94f-40c4a90b8a54 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:21.027976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2a340-e289-4661-8b38-08feb42a6e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:21.029472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2a340-e289-4661-8b38-08feb42a6e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.030204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2a340-e289-4661-8b38-08feb42a6e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.030344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ec9c0-9310-47be-b6f9-ea2d7a43a8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:21.031126 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ec9c0-9310-47be-b6f9-ea2d7a43a8cf admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.031417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ec9c0-9310-47be-b6f9-ea2d7a43a8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.031748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ec9c0-9310-47be-b6f9-ea2d7a43a8cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.031968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2a340-e289-4661-8b38-08feb42a6e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.033756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2a340-e289-4661-8b38-08feb42a6e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:21.034860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ec9c0-9310-47be-b6f9-ea2d7a43a8cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:21.035125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ec9c0-9310-47be-b6f9-ea2d7a43a8cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:21.042375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ec9c0-9310-47be-b6f9-ea2d7a43a8cf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:21.055103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2a340-e289-4661-8b38-08feb42a6e48 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:21.059327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 849/1716] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:21.064582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ec9c0-9310-47be-b6f9-ea2d7a43a8cf admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:21.077085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ec9c0-9310-47be-b6f9-ea2d7a43a8cf admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:21.083229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744d0bd-7bbd-4c03-8652-6fec2c0a2d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:21.084155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744d0bd-7bbd-4c03-8652-6fec2c0a2d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.084470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744d0bd-7bbd-4c03-8652-6fec2c0a2d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.084748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744d0bd-7bbd-4c03-8652-6fec2c0a2d14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.085466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744d0bd-7bbd-4c03-8652-6fec2c0a2d14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:21.085791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744d0bd-7bbd-4c03-8652-6fec2c0a2d14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:21.091810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 868/1717] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:21.094349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2a340-e289-4661-8b38-08feb42a6e48 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:21.113858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744d0bd-7bbd-4c03-8652-6fec2c0a2d14 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:21.160122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2a340-e289-4661-8b38-08feb42a6e48 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:21.174377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 850/1718] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:21.175354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744d0bd-7bbd-4c03-8652-6fec2c0a2d14 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:21.203823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8330ae5-18ab-4db2-b7cc-1d215e0ca72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:21.204933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8330ae5-18ab-4db2-b7cc-1d215e0ca72e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.205407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8330ae5-18ab-4db2-b7cc-1d215e0ca72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.205895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8330ae5-18ab-4db2-b7cc-1d215e0ca72e admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.207165 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8330ae5-18ab-4db2-b7cc-1d215e0ca72e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:21.218402 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8330ae5-18ab-4db2-b7cc-1d215e0ca72e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:21.221208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744d0bd-7bbd-4c03-8652-6fec2c0a2d14 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:21.238223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb8a98-3b2b-40ec-991d-ca53481f75f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:21.239056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb8a98-3b2b-40ec-991d-ca53481f75f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.239373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb8a98-3b2b-40ec-991d-ca53481f75f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.250820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb8a98-3b2b-40ec-991d-ca53481f75f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.251604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb8a98-3b2b-40ec-991d-ca53481f75f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:21.255870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb8a98-3b2b-40ec-991d-ca53481f75f3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:21.290811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 851/1719] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:21.292352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb8a98-3b2b-40ec-991d-ca53481f75f3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:21.310201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb8a98-3b2b-40ec-991d-ca53481f75f3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:21.318676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc18ba6-99fa-405f-8c79-d155018d7ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:21.320303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc18ba6-99fa-405f-8c79-d155018d7ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.320624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc18ba6-99fa-405f-8c79-d155018d7ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.321140 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc18ba6-99fa-405f-8c79-d155018d7ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.322135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc18ba6-99fa-405f-8c79-d155018d7ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:21.325484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc18ba6-99fa-405f-8c79-d155018d7ff5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:21.335147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 869/1720] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:21 2022] POST /identity/v3/projects => generated 405 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:21.336234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8330ae5-18ab-4db2-b7cc-1d215e0ca72e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:21.432186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 852/1721] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:21 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:21.438936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc18ba6-99fa-405f-8c79-d155018d7ff5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:21.541713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8330ae5-18ab-4db2-b7cc-1d215e0ca72e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:21.554886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:21.594667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc18ba6-99fa-405f-8c79-d155018d7ff5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:21.601609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1c7d5-3dca-49aa-b91a-158dded2497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:21.602382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1c7d5-3dca-49aa-b91a-158dded2497a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.602535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1c7d5-3dca-49aa-b91a-158dded2497a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.602810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1c7d5-3dca-49aa-b91a-158dded2497a admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.603689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1c7d5-3dca-49aa-b91a-158dded2497a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:21.624574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1c7d5-3dca-49aa-b91a-158dded2497a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:21.667388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 853/1722] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:21 2022] POST /identity/v3/users => generated 303 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:21.683936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40090d40-d2b7-4927-b902-6583f741b87c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.684319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40090d40-d2b7-4927-b902-6583f741b87c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.684622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40090d40-d2b7-4927-b902-6583f741b87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.707659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-40090d40-d2b7-4927-b902-6583f741b87c None None] MFA Rules not processed for user `218f3dd17e6549fc8796eb30e4b48e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:21.709366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-40090d40-d2b7-4927-b902-6583f741b87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:21.816290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297fb9-a8c2-48bf-8c13-d36a9afab5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:21.818972 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297fb9-a8c2-48bf-8c13-d36a9afab5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.820821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297fb9-a8c2-48bf-8c13-d36a9afab5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.821187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297fb9-a8c2-48bf-8c13-d36a9afab5ef admin admin] PATH_INFO: `/v3/projects/56160baf00fb4e7c916b6ca6232f4fac/users/338d6ec4042748b49c75a67de95cb645/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.832592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 854/1723] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:21.833409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40090d40-d2b7-4927-b902-6583f741b87c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:21.857386 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40090d40-d2b7-4927-b902-6583f741b87c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:21.875366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795574b1-e73c-48aa-9373-1ff58550fc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:21.877619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795574b1-e73c-48aa-9373-1ff58550fc2b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.878785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795574b1-e73c-48aa-9373-1ff58550fc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.878785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795574b1-e73c-48aa-9373-1ff58550fc2b admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:21.883305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795574b1-e73c-48aa-9373-1ff58550fc2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:21.885402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795574b1-e73c-48aa-9373-1ff58550fc2b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:21.912179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15297fb9-a8c2-48bf-8c13-d36a9afab5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=56160baf00fb4e7c916b6ca6232f4fac, user_id=338d6ec4042748b49c75a67de95cb645, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:21.924742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15297fb9-a8c2-48bf-8c13-d36a9afab5ef admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:21.934075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:21.965964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 855/1724] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:21 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:21.965964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf5e0a-3187-4111-9c2c-6c79eeaa9061 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:21.965964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf5e0a-3187-4111-9c2c-6c79eeaa9061 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:21.966278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf5e0a-3187-4111-9c2c-6c79eeaa9061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:22.023918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4baf5e0a-3187-4111-9c2c-6c79eeaa9061 None None] MFA Rules not processed for user `dc2bbdceb12641e6a0595e68f7cefd63`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:22.025567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4baf5e0a-3187-4111-9c2c-6c79eeaa9061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:22.029948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 870/1725] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:21 2022] PUT /identity/v3/projects/56160baf00fb4e7c916b6ca6232f4fac/users/338d6ec4042748b49c75a67de95cb645/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:22.031587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297fb9-a8c2-48bf-8c13-d36a9afab5ef admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:22.203516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 856/1726] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:22.205930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf5e0a-3187-4111-9c2c-6c79eeaa9061 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:22.293769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297fb9-a8c2-48bf-8c13-d36a9afab5ef admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:22.314707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644521b6-dbf7-460d-a44f-6102b96026cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:22.315390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644521b6-dbf7-460d-a44f-6102b96026cc admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:22.315678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644521b6-dbf7-460d-a44f-6102b96026cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:22.315953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644521b6-dbf7-460d-a44f-6102b96026cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:22.316644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644521b6-dbf7-460d-a44f-6102b96026cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:22.318279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644521b6-dbf7-460d-a44f-6102b96026cc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:22.364258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 871/1727] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:22.365238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6a9b1-6be2-4a3f-801d-90c168f90772 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:22.365533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6a9b1-6be2-4a3f-801d-90c168f90772 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:22.365809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6a9b1-6be2-4a3f-801d-90c168f90772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:22.435268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf5e0a-3187-4111-9c2c-6c79eeaa9061 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:22.457259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b4b3c-87c1-43b0-9ea3-a5cb6203a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:22.457849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b4b3c-87c1-43b0-9ea3-a5cb6203a287 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:22.458062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b4b3c-87c1-43b0-9ea3-a5cb6203a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:22.458306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b4b3c-87c1-43b0-9ea3-a5cb6203a287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:22.458960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b4b3c-87c1-43b0-9ea3-a5cb6203a287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:22.460635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b4b3c-87c1-43b0-9ea3-a5cb6203a287 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:22.461434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2ce6a9b1-6be2-4a3f-801d-90c168f90772 None None] MFA Rules not processed for user `338d6ec4042748b49c75a67de95cb645`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:22.462924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2ce6a9b1-6be2-4a3f-801d-90c168f90772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:22.494563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 857/1728] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:22.498410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21358848-c608-4876-b6b6-5d217beb47cf None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:22.498732 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21358848-c608-4876-b6b6-5d217beb47cf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:22.499008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21358848-c608-4876-b6b6-5d217beb47cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:22.551662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-21358848-c608-4876-b6b6-5d217beb47cf None None] MFA Rules not processed for user `4aaa94a183fb49b79e6383ea783ab76d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:22.553130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-21358848-c608-4876-b6b6-5d217beb47cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:22.657498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 872/1729] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:22.660087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6a9b1-6be2-4a3f-801d-90c168f90772 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:22.675431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6a9b1-6be2-4a3f-801d-90c168f90772 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:22.686471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fbda4-f424-4b50-93b2-722f92237a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:22.687202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fbda4-f424-4b50-93b2-722f92237a4a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:22.687703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fbda4-f424-4b50-93b2-722f92237a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:22.688305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fbda4-f424-4b50-93b2-722f92237a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:22.689883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fbda4-f424-4b50-93b2-722f92237a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:22.691763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fbda4-f424-4b50-93b2-722f92237a4a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:22.710437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 873/1730] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:22.712625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fbda4-f424-4b50-93b2-722f92237a4a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:22.725550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fbda4-f424-4b50-93b2-722f92237a4a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:22.732423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57252a7-8bf5-4838-8062-68cfc2648484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:22.733609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57252a7-8bf5-4838-8062-68cfc2648484 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:22.734091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57252a7-8bf5-4838-8062-68cfc2648484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:22.734675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57252a7-8bf5-4838-8062-68cfc2648484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:22.735719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57252a7-8bf5-4838-8062-68cfc2648484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:22.737820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57252a7-8bf5-4838-8062-68cfc2648484 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:22.738729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 858/1731] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:22.739693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21358848-c608-4876-b6b6-5d217beb47cf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:22.740668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:22.760790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 874/1732] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:22.763137 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57252a7-8bf5-4838-8062-68cfc2648484 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:22.767446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21358848-c608-4876-b6b6-5d217beb47cf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:22.778111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57252a7-8bf5-4838-8062-68cfc2648484 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:22.782237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdd857-a974-46a9-8881-9ff52d69f93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:22.782869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdd857-a974-46a9-8881-9ff52d69f93e admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:22.783150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdd857-a974-46a9-8881-9ff52d69f93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:22.783422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdd857-a974-46a9-8881-9ff52d69f93e admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:22.784054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdd857-a974-46a9-8881-9ff52d69f93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:22.786261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdd857-a974-46a9-8881-9ff52d69f93e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:22.789788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8e5fa-0df2-4484-a20c-d918682d136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:22.791428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8e5fa-0df2-4484-a20c-d918682d136c admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:22.792792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8e5fa-0df2-4484-a20c-d918682d136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:22.793497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8e5fa-0df2-4484-a20c-d918682d136c admin admin] PATH_INFO: `/v3/projects/4b43df32153c4ef6bfb8e65c3fb4f098/users/dd6750d7c78b4d9bbe8e55c784fa3ff6/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:22.820888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8e5fa-0df2-4484-a20c-d918682d136c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b43df32153c4ef6bfb8e65c3fb4f098, user_id=dd6750d7c78b4d9bbe8e55c784fa3ff6, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:22.825999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8e5fa-0df2-4484-a20c-d918682d136c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:22.864691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 875/1733] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:22 2022] PUT /identity/v3/projects/4b43df32153c4ef6bfb8e65c3fb4f098/users/dd6750d7c78b4d9bbe8e55c784fa3ff6/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:22.865969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8e5fa-0df2-4484-a20c-d918682d136c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:22.875164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 859/1734] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:22 2022] POST /identity/v3/projects => generated 369 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:22.876985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdd857-a974-46a9-8881-9ff52d69f93e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:22.945030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:23.014943 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8e5fa-0df2-4484-a20c-d918682d136c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:23.045935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdd857-a974-46a9-8881-9ff52d69f93e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:23.064718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04910c48-4613-4b41-ab5b-47dc3488263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:23.065504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04910c48-4613-4b41-ab5b-47dc3488263c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:23.065760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04910c48-4613-4b41-ab5b-47dc3488263c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:23.066039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04910c48-4613-4b41-ab5b-47dc3488263c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:23.066698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04910c48-4613-4b41-ab5b-47dc3488263c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:23.069429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04910c48-4613-4b41-ab5b-47dc3488263c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:23.101496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 860/1735] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:22 2022] POST /identity/v3/users => generated 291 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:23.107677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea743f93-4edf-427a-b448-e461e7a9b527 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:23.107974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea743f93-4edf-427a-b448-e461e7a9b527 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:23.108421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea743f93-4edf-427a-b448-e461e7a9b527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:23.116648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461cb47-118c-4f27-b62b-a85861db907b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:23.119673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461cb47-118c-4f27-b62b-a85861db907b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:23.120931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461cb47-118c-4f27-b62b-a85861db907b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:23.122520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461cb47-118c-4f27-b62b-a85861db907b admin admin] PATH_INFO: `/v3/projects/0c194bbe8007427cb9b45940d9e6ad87/users/9aa29f1a9d044e92bbfdb2964a992c6d/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:23.145606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ea743f93-4edf-427a-b448-e461e7a9b527 None None] MFA Rules not processed for user `dd6750d7c78b4d9bbe8e55c784fa3ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:23.146860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ea743f93-4edf-427a-b448-e461e7a9b527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:23.161021 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461cb47-118c-4f27-b62b-a85861db907b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c194bbe8007427cb9b45940d9e6ad87, user_id=9aa29f1a9d044e92bbfdb2964a992c6d, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:23.169342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461cb47-118c-4f27-b62b-a85861db907b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:23.208814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 876/1736] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:22 2022] PUT /identity/v3/projects/0c194bbe8007427cb9b45940d9e6ad87/users/9aa29f1a9d044e92bbfdb2964a992c6d/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:23.210293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461cb47-118c-4f27-b62b-a85861db907b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:23.247084 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 861/1737] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:23 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:23.248506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc4bdf-3d5d-40b7-8def-815722662a27 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:23.249208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc4bdf-3d5d-40b7-8def-815722662a27 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:23.249671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc4bdf-3d5d-40b7-8def-815722662a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:23.310752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-debc4bdf-3d5d-40b7-8def-815722662a27 None None] MFA Rules not processed for user `9aa29f1a9d044e92bbfdb2964a992c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:23.311987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-debc4bdf-3d5d-40b7-8def-815722662a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:23.348446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461cb47-118c-4f27-b62b-a85861db907b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:23.362089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbb99c-3614-4ff5-a9eb-0cde64b9a14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:23.362721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbb99c-3614-4ff5-a9eb-0cde64b9a14d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:23.363037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbb99c-3614-4ff5-a9eb-0cde64b9a14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:23.363265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbb99c-3614-4ff5-a9eb-0cde64b9a14d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:23.364086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbb99c-3614-4ff5-a9eb-0cde64b9a14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:23.365758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbb99c-3614-4ff5-a9eb-0cde64b9a14d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:23.370652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:23.391397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 877/1738] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:23.392510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbb99c-3614-4ff5-a9eb-0cde64b9a14d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:23.413563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbb99c-3614-4ff5-a9eb-0cde64b9a14d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:23.427074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7784be1-78b3-48f6-a149-ff0238145c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:23.427701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7784be1-78b3-48f6-a149-ff0238145c0e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:23.427978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7784be1-78b3-48f6-a149-ff0238145c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:23.428253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7784be1-78b3-48f6-a149-ff0238145c0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:23.429559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7784be1-78b3-48f6-a149-ff0238145c0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:23.438440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7784be1-78b3-48f6-a149-ff0238145c0e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:23.470824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 862/1739] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:23.472237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc4bdf-3d5d-40b7-8def-815722662a27 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:23.501764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 878/1740] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:23 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:23.502983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7784be1-78b3-48f6-a149-ff0238145c0e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:23.514873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc4bdf-3d5d-40b7-8def-815722662a27 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:23.593787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:23.658351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7784be1-78b3-48f6-a149-ff0238145c0e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:23.672520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa2294-deea-4867-9654-1135b4eb02e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:23.673280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa2294-deea-4867-9654-1135b4eb02e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:23.673701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa2294-deea-4867-9654-1135b4eb02e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:23.674078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa2294-deea-4867-9654-1135b4eb02e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:23.674844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa2294-deea-4867-9654-1135b4eb02e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:23.685648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa2294-deea-4867-9654-1135b4eb02e2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:23.757420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2d2da-34f7-43b3-8e8a-6140c6c0a18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:23.758471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d2da-34f7-43b3-8e8a-6140c6c0a18a admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:23.758761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d2da-34f7-43b3-8e8a-6140c6c0a18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:23.759632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d2da-34f7-43b3-8e8a-6140c6c0a18a admin admin] PATH_INFO: `/v3/projects/55f1e21aa0c7480bab768655e10bb612/users/7bc0fa0909c6484aa393c67f9d299805/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:23.786410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2d2da-34f7-43b3-8e8a-6140c6c0a18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f1e21aa0c7480bab768655e10bb612, user_id=7bc0fa0909c6484aa393c67f9d299805, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:23.787869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 879/1741] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:23 2022] POST /identity/v3/projects => generated 367 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:23.790122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2d2da-34f7-43b3-8e8a-6140c6c0a18a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:23.791179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa2294-deea-4867-9654-1135b4eb02e2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:23.841854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 863/1742] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:23 2022] PUT /identity/v3/projects/55f1e21aa0c7480bab768655e10bb612/users/7bc0fa0909c6484aa393c67f9d299805/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:23.845295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2d2da-34f7-43b3-8e8a-6140c6c0a18a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:24.079818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2d2da-34f7-43b3-8e8a-6140c6c0a18a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:24.102047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9a934-bd8d-452c-a93f-c15e06022b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:24.102740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9a934-bd8d-452c-a93f-c15e06022b64 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:24.103030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9a934-bd8d-452c-a93f-c15e06022b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:24.107217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9a934-bd8d-452c-a93f-c15e06022b64 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:24.107954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9a934-bd8d-452c-a93f-c15e06022b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:24.112218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9a934-bd8d-452c-a93f-c15e06022b64 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:24.178366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa2294-deea-4867-9654-1135b4eb02e2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:24.191798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e52bff-4390-494f-a543-1f11413b055b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:24.192599 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e52bff-4390-494f-a543-1f11413b055b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:24.192980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e52bff-4390-494f-a543-1f11413b055b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:24.193355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e52bff-4390-494f-a543-1f11413b055b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:24.194101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e52bff-4390-494f-a543-1f11413b055b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:24.196839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e52bff-4390-494f-a543-1f11413b055b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:24.201209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 864/1743] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:23 2022] POST /identity/v3/users => generated 290 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:24.202123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9a934-bd8d-452c-a93f-c15e06022b64 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:24.254439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9a934-bd8d-452c-a93f-c15e06022b64 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:24.254811 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 880/1744] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:23 2022] POST /identity/v3/users => generated 309 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:24.259761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e52bff-4390-494f-a543-1f11413b055b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:24.279085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ca3d7-0f60-44b6-ae2a-1410fff09443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:24.280065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ca3d7-0f60-44b6-ae2a-1410fff09443 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:24.280359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ca3d7-0f60-44b6-ae2a-1410fff09443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:24.280673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ca3d7-0f60-44b6-ae2a-1410fff09443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:24.282092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ca3d7-0f60-44b6-ae2a-1410fff09443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:24.287555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ca3d7-0f60-44b6-ae2a-1410fff09443 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:24.316157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e52bff-4390-494f-a543-1f11413b055b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:24.324626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 865/1745] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:24.325597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ca3d7-0f60-44b6-ae2a-1410fff09443 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:24.337848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ca3d7-0f60-44b6-ae2a-1410fff09443 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:24.339127 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c8127-9349-4558-8183-00a9d51f3ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:24.340733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c8127-9349-4558-8183-00a9d51f3ebf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:24.341354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c8127-9349-4558-8183-00a9d51f3ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:24.342868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c8127-9349-4558-8183-00a9d51f3ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:24.343977 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c5d71-0b10-4cc1-b440-0bf794adfc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:24.344698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c5d71-0b10-4cc1-b440-0bf794adfc39 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:24.344955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c5d71-0b10-4cc1-b440-0bf794adfc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:24.345231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c5d71-0b10-4cc1-b440-0bf794adfc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:24.345898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c5d71-0b10-4cc1-b440-0bf794adfc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:24.346400 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c8127-9349-4558-8183-00a9d51f3ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:24.347613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c5d71-0b10-4cc1-b440-0bf794adfc39 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:24.368359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 866/1746] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:24.372033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c5d71-0b10-4cc1-b440-0bf794adfc39 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:24.376374 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c8127-9349-4558-8183-00a9d51f3ebf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:24.384454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c5d71-0b10-4cc1-b440-0bf794adfc39 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:24.390921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7722f-4472-409f-a12e-b62ef55fb01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:24.391789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7722f-4472-409f-a12e-b62ef55fb01d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:24.392008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7722f-4472-409f-a12e-b62ef55fb01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:24.392290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7722f-4472-409f-a12e-b62ef55fb01d admin admin] PATH_INFO: `/v3/projects/55f1e21aa0c7480bab768655e10bb612/users/7bc0fa0909c6484aa393c67f9d299805/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:24.395541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7722f-4472-409f-a12e-b62ef55fb01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f1e21aa0c7480bab768655e10bb612, user_id=7bc0fa0909c6484aa393c67f9d299805, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:24.399083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7722f-4472-409f-a12e-b62ef55fb01d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:24.418177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 867/1747] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:24 2022] PUT /identity/v3/projects/55f1e21aa0c7480bab768655e10bb612/users/7bc0fa0909c6484aa393c67f9d299805/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:24.419284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7722f-4472-409f-a12e-b62ef55fb01d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:24.439901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 881/1748] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:24.441738 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbce85-e143-4d7b-ab00-97f5d6a9adfc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:24.444254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbce85-e143-4d7b-ab00-97f5d6a9adfc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:24.444774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbce85-e143-4d7b-ab00-97f5d6a9adfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:24.499982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-48fbce85-e143-4d7b-ab00-97f5d6a9adfc None None] MFA Rules not processed for user `7bc0fa0909c6484aa393c67f9d299805`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:24.501704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-48fbce85-e143-4d7b-ab00-97f5d6a9adfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:24.641800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:24.656047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7722f-4472-409f-a12e-b62ef55fb01d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:24.672811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132f9cb-eed0-4be4-9435-9a11b0bb57c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:24.672811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132f9cb-eed0-4be4-9435-9a11b0bb57c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:24.673919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132f9cb-eed0-4be4-9435-9a11b0bb57c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:24.676351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132f9cb-eed0-4be4-9435-9a11b0bb57c7 admin admin] PATH_INFO: `/v3/projects/397314a16ec54459a03d5338cdda0d34/users/540f8c251d8d413aa3a55fa7ffd3c6fe/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:24.713145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:24.731558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132f9cb-eed0-4be4-9435-9a11b0bb57c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=397314a16ec54459a03d5338cdda0d34, user_id=540f8c251d8d413aa3a55fa7ffd3c6fe, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:24.740520 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132f9cb-eed0-4be4-9435-9a11b0bb57c7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:24.775649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 868/1749] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:24 2022] PUT /identity/v3/projects/397314a16ec54459a03d5338cdda0d34/users/540f8c251d8d413aa3a55fa7ffd3c6fe/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:24.780378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132f9cb-eed0-4be4-9435-9a11b0bb57c7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:24.847982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 882/1750] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:24.856615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbce85-e143-4d7b-ab00-97f5d6a9adfc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:24.874580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbce85-e143-4d7b-ab00-97f5d6a9adfc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:24.962231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132f9cb-eed0-4be4-9435-9a11b0bb57c7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:24.974370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1dfe3-5a40-4b78-ae92-56c6d1abc3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:24.975062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1dfe3-5a40-4b78-ae92-56c6d1abc3e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:24.975350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1dfe3-5a40-4b78-ae92-56c6d1abc3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:24.975625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1dfe3-5a40-4b78-ae92-56c6d1abc3e9 admin admin] PATH_INFO: `/v3/projects/57bf2744971d4d67b602bf5798fe9627/users/b9a347cfaf0443c3a4d27afc9666ffcf/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:25.071990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1dfe3-5a40-4b78-ae92-56c6d1abc3e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bf2744971d4d67b602bf5798fe9627, user_id=b9a347cfaf0443c3a4d27afc9666ffcf, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:25.083465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af7bc3-95e5-49d5-9730-c8a3e1810a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:25.084103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1dfe3-5a40-4b78-ae92-56c6d1abc3e9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:25.086367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af7bc3-95e5-49d5-9730-c8a3e1810a08 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:25.087838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af7bc3-95e5-49d5-9730-c8a3e1810a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:25.088335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af7bc3-95e5-49d5-9730-c8a3e1810a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:25.089272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af7bc3-95e5-49d5-9730-c8a3e1810a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:25.100844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af7bc3-95e5-49d5-9730-c8a3e1810a08 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:25.131979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 883/1751] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:24 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:25.135899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af7bc3-95e5-49d5-9730-c8a3e1810a08 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:25.144013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a6af7bc3-95e5-49d5-9730-c8a3e1810a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:25.150030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 869/1752] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:24 2022] PUT /identity/v3/projects/57bf2744971d4d67b602bf5798fe9627/users/b9a347cfaf0443c3a4d27afc9666ffcf/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:25.158094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1dfe3-5a40-4b78-ae92-56c6d1abc3e9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:25.305294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1dfe3-5a40-4b78-ae92-56c6d1abc3e9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:25.313072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557ac1d-9308-47d9-a508-6bd938202f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:25.314008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557ac1d-9308-47d9-a508-6bd938202f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:25.314585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557ac1d-9308-47d9-a508-6bd938202f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:25.314967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557ac1d-9308-47d9-a508-6bd938202f3d admin admin] PATH_INFO: `/v3/projects/397314a16ec54459a03d5338cdda0d34/users/540f8c251d8d413aa3a55fa7ffd3c6fe/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:25.317839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557ac1d-9308-47d9-a508-6bd938202f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=397314a16ec54459a03d5338cdda0d34, user_id=540f8c251d8d413aa3a55fa7ffd3c6fe, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:25.323100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557ac1d-9308-47d9-a508-6bd938202f3d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:25.361560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af7bc3-95e5-49d5-9730-c8a3e1810a08 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:25.362068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 870/1753] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:25 2022] PUT /identity/v3/projects/397314a16ec54459a03d5338cdda0d34/users/540f8c251d8d413aa3a55fa7ffd3c6fe/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:25.362631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557ac1d-9308-47d9-a508-6bd938202f3d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:25.559635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a5b02-a50f-4630-b25e-10bec2d316e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:25.565542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a5b02-a50f-4630-b25e-10bec2d316e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:25.571628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a5b02-a50f-4630-b25e-10bec2d316e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:25.572482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a5b02-a50f-4630-b25e-10bec2d316e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:25.627065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557ac1d-9308-47d9-a508-6bd938202f3d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:25.641566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ef704-4daf-41f4-8b8c-d59cf72ddba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:25.642465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ef704-4daf-41f4-8b8c-d59cf72ddba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:25.642465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ef704-4daf-41f4-8b8c-d59cf72ddba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:25.642670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ef704-4daf-41f4-8b8c-d59cf72ddba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:25.643218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ef704-4daf-41f4-8b8c-d59cf72ddba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:25.644910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ef704-4daf-41f4-8b8c-d59cf72ddba9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:25.714899 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 871/1754] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:25.717176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c86550-960b-49b1-9f2a-d56856da2adf None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:25.722566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c86550-960b-49b1-9f2a-d56856da2adf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:25.722566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c86550-960b-49b1-9f2a-d56856da2adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:25.742107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:25.753179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:25.837522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c4c86550-960b-49b1-9f2a-d56856da2adf None None] MFA Rules not processed for user `540f8c251d8d413aa3a55fa7ffd3c6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:25.844328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c4c86550-960b-49b1-9f2a-d56856da2adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:25.878151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a5b02-a50f-4630-b25e-10bec2d316e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:25.880098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a5b02-a50f-4630-b25e-10bec2d316e7 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:25.940875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 884/1755] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 808 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:25.944415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a5b02-a50f-4630-b25e-10bec2d316e7 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:25.995448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a5b02-a50f-4630-b25e-10bec2d316e7 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:26.020999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb5146-c931-4704-a0f0-72a9f7dc8a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:26.022211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb5146-c931-4704-a0f0-72a9f7dc8a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:26.023006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb5146-c931-4704-a0f0-72a9f7dc8a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:26.023396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb5146-c931-4704-a0f0-72a9f7dc8a15 admin admin] PATH_INFO: `/v3/projects/57bf2744971d4d67b602bf5798fe9627/users/b9a347cfaf0443c3a4d27afc9666ffcf/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:26.032856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb5146-c931-4704-a0f0-72a9f7dc8a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57bf2744971d4d67b602bf5798fe9627, user_id=b9a347cfaf0443c3a4d27afc9666ffcf, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:26.059858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb5146-c931-4704-a0f0-72a9f7dc8a15 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:26.075325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 872/1756] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:26.087433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c86550-960b-49b1-9f2a-d56856da2adf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:26.101222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c86550-960b-49b1-9f2a-d56856da2adf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:26.126045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 885/1757] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:25 2022] PUT /identity/v3/projects/57bf2744971d4d67b602bf5798fe9627/users/b9a347cfaf0443c3a4d27afc9666ffcf/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:26.141535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0b7e5-5c27-4964-8380-b9a2b808042a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:26.141976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0b7e5-5c27-4964-8380-b9a2b808042a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:26.142377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0b7e5-5c27-4964-8380-b9a2b808042a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:26.212399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2ccb0-3cbe-4061-877f-3eeff2691010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:26.213561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2ccb0-3cbe-4061-877f-3eeff2691010 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:26.217597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2ccb0-3cbe-4061-877f-3eeff2691010 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:26.217896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2ccb0-3cbe-4061-877f-3eeff2691010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:26.239764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-69c0b7e5-5c27-4964-8380-b9a2b808042a None None] MFA Rules not processed for user `b9a347cfaf0443c3a4d27afc9666ffcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:26.244128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-69c0b7e5-5c27-4964-8380-b9a2b808042a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:26.390639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2ccb0-3cbe-4061-877f-3eeff2691010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:26.392151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2ccb0-3cbe-4061-877f-3eeff2691010 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:26.423016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 873/1758] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:26 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:26.454501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 886/1759] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:26.496043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2ccb0-3cbe-4061-877f-3eeff2691010 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:26.498835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ceb2ccb0-3cbe-4061-877f-3eeff2691010 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:26.537068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2ccb0-3cbe-4061-877f-3eeff2691010 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:26.550706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051feebc-3945-4a3f-91be-a4d60171ad12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:26.552363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051feebc-3945-4a3f-91be-a4d60171ad12 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:26.552363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051feebc-3945-4a3f-91be-a4d60171ad12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:26.552363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051feebc-3945-4a3f-91be-a4d60171ad12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:26.579335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051feebc-3945-4a3f-91be-a4d60171ad12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:26.584132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051feebc-3945-4a3f-91be-a4d60171ad12 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:26.625034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 874/1760] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:26.982446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b8d13-b01b-4382-b350-8a3bc1ef32fc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:26.982752 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b8d13-b01b-4382-b350-8a3bc1ef32fc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:26.983000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b8d13-b01b-4382-b350-8a3bc1ef32fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:26.986549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051feebc-3945-4a3f-91be-a4d60171ad12 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:27.000363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051feebc-3945-4a3f-91be-a4d60171ad12 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:27.028312 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0fac6-c247-49cb-9530-b41370fd3249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:27.028973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0fac6-c247-49cb-9530-b41370fd3249 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:27.029254 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0fac6-c247-49cb-9530-b41370fd3249 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:27.029528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0fac6-c247-49cb-9530-b41370fd3249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:27.118229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:27.202779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6c9b8d13-b01b-4382-b350-8a3bc1ef32fc None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:27.206311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6c9b8d13-b01b-4382-b350-8a3bc1ef32fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:27.281845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0fac6-c247-49cb-9530-b41370fd3249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:27.283346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0fac6-c247-49cb-9530-b41370fd3249 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:27.327260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 875/1761] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:27.426145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 887/1762] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:23:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:27.498905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0fac6-c247-49cb-9530-b41370fd3249 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:27.773582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0fac6-c247-49cb-9530-b41370fd3249 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:27.795363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:27.795363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:27.795363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:27.795363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] PATH_INFO: `/v3/users/dd6750d7c78b4d9bbe8e55c784fa3ff6` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:27.796056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd6750d7c78b4d9bbe8e55c784fa3ff6)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:27.796056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:27.886298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b8d13-b01b-4382-b350-8a3bc1ef32fc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:27.888640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6c9b8d13-b01b-4382-b350-8a3bc1ef32fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:27.937276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:27.997044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd6750d7c78b4d9bbe8e55c784fa3ff6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:28.020970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd6750d7c78b4d9bbe8e55c784fa3ff6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:28.025286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd6750d7c78b4d9bbe8e55c784fa3ff6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:28.057256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd6750d7c78b4d9bbe8e55c784fa3ff6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:28.083464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d8ad5512-3971-4ce9-8c71-d7d26b532ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd6750d7c78b4d9bbe8e55c784fa3ff6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:28.097490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 876/1763] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:27 2022] DELETE /identity/v3/users/dd6750d7c78b4d9bbe8e55c784fa3ff6 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:28.100188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee1ce2-4ff7-4ca4-bd37-b9c7397e1dc7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:28.100571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee1ce2-4ff7-4ca4-bd37-b9c7397e1dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:28.100783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee1ce2-4ff7-4ca4-bd37-b9c7397e1dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:28.173504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-43ee1ce2-4ff7-4ca4-bd37-b9c7397e1dc7 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:28.175050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-43ee1ce2-4ff7-4ca4-bd37-b9c7397e1dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:28.221136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b8d13-b01b-4382-b350-8a3bc1ef32fc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:28.359971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:28.386969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:28.389142 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:28.391523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:28.392031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:28.395837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:28.457176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 877/1764] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:28.516517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:28.519365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:28.532314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 888/1765] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:23:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 15:23:29.054404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee1ce2-4ff7-4ca4-bd37-b9c7397e1dc7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:29.056778 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-43ee1ce2-4ff7-4ca4-bd37-b9c7397e1dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:29.263787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee1ce2-4ff7-4ca4-bd37-b9c7397e1dc7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:29.273748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b1f54-946b-48ee-a23a-5e3ccfa9b5fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:29.274522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b1f54-946b-48ee-a23a-5e3ccfa9b5fc service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:29.274873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b1f54-946b-48ee-a23a-5e3ccfa9b5fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:29.275213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b1f54-946b-48ee-a23a-5e3ccfa9b5fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:29.300145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b1f54-946b-48ee-a23a-5e3ccfa9b5fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:29.306362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b1f54-946b-48ee-a23a-5e3ccfa9b5fc service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:29.351294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 878/1766] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:30.901245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:30.902871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:30.933982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017668f-e2ca-43f0-b3b2-5fa0ab928ccc service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:30.940726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:30.941738 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:30.942213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:30.942701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] PATH_INFO: `/v3/users/338d6ec4042748b49c75a67de95cb645` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:30.944598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] RBAC: Authorizing `identity:delete_user(user_id=338d6ec4042748b49c75a67de95cb645)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:30.946894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:30.987390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b1f54-946b-48ee-a23a-5e3ccfa9b5fc service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:30.988998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-229b1f54-946b-48ee-a23a-5e3ccfa9b5fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:31.003107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:31.005017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b1f54-946b-48ee-a23a-5e3ccfa9b5fc service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:31.017586 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:31.018225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:31.018499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:31.018913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] PATH_INFO: `/v3/projects/4b43df32153c4ef6bfb8e65c3fb4f098` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:31.020375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b43df32153c4ef6bfb8e65c3fb4f098)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:31.023449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:31.027644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338d6ec4042748b49c75a67de95cb645'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.028498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338d6ec4042748b49c75a67de95cb645'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.029208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338d6ec4042748b49c75a67de95cb645'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.038575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338d6ec4042748b49c75a67de95cb645'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.082235 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fcc2ff4e-849c-49c8-a52a-7f56c3fd135e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338d6ec4042748b49c75a67de95cb645'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.113429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 889/1767] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:30 2022] DELETE /identity/v3/users/338d6ec4042748b49c75a67de95cb645 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:31.117499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91fa9-eb4b-4861-b22b-e3e15d8ad24e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:31.118194 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91fa9-eb4b-4861-b22b-e3e15d8ad24e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:31.118932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91fa9-eb4b-4861-b22b-e3e15d8ad24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:31.154825 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b43df32153c4ef6bfb8e65c3fb4f098', 'request_id': 'req-a566f980-8f6c-4761-9cdf-158a5b7150fb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.156096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a7c91fa9-eb4b-4861-b22b-e3e15d8ad24e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:31.158092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a7c91fa9-eb4b-4861-b22b-e3e15d8ad24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:31.199005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b43df32153c4ef6bfb8e65c3fb4f098', 'request_id': 'req-a566f980-8f6c-4761-9cdf-158a5b7150fb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.219730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b43df32153c4ef6bfb8e65c3fb4f098', 'request_id': 'req-a566f980-8f6c-4761-9cdf-158a5b7150fb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.240129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] The token cache is being invalidate because project 4b43df32153c4ef6bfb8e65c3fb4f098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:31.241357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.242112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.245853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 879/1768] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:30 2022] DELETE /identity/v3/projects/4b43df32153c4ef6bfb8e65c3fb4f098 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:31.250071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:31.251839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:31.317946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 890/1769] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:31.331349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c91fa9-eb4b-4861-b22b-e3e15d8ad24e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:31.333159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a7c91fa9-eb4b-4861-b22b-e3e15d8ad24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:31.396654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566f980-8f6c-4761-9cdf-158a5b7150fb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:31.405144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:31.405793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:31.406076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:31.406350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] PATH_INFO: `/v3/users/540f8c251d8d413aa3a55fa7ffd3c6fe` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:31.409617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=540f8c251d8d413aa3a55fa7ffd3c6fe)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:31.411591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:31.443232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:31.472621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c91fa9-eb4b-4861-b22b-e3e15d8ad24e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:31.481358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0900584-0fa8-4d4a-9aa4-6a84f97338f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:31.482375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0900584-0fa8-4d4a-9aa4-6a84f97338f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:31.482658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0900584-0fa8-4d4a-9aa4-6a84f97338f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:31.482941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0900584-0fa8-4d4a-9aa4-6a84f97338f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:31.517761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0900584-0fa8-4d4a-9aa4-6a84f97338f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:31.520433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0900584-0fa8-4d4a-9aa4-6a84f97338f0 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:31.542686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '540f8c251d8d413aa3a55fa7ffd3c6fe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.548592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 891/1770] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:31.577854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '540f8c251d8d413aa3a55fa7ffd3c6fe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.578881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '540f8c251d8d413aa3a55fa7ffd3c6fe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.607563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '540f8c251d8d413aa3a55fa7ffd3c6fe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.626065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '540f8c251d8d413aa3a55fa7ffd3c6fe'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:31.637525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 880/1771] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:31 2022] DELETE /identity/v3/users/540f8c251d8d413aa3a55fa7ffd3c6fe => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:32.283509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065d2c3-5ca4-4179-8bcc-a16ab24c263e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:32.284065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065d2c3-5ca4-4179-8bcc-a16ab24c263e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:32.284340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065d2c3-5ca4-4179-8bcc-a16ab24c263e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:32.313117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e065d2c3-5ca4-4179-8bcc-a16ab24c263e None None] MFA Rules not processed for user `c8704aae85084d41bdedb5537ef96e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:32.315416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e065d2c3-5ca4-4179-8bcc-a16ab24c263e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:32.384792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:32.398567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 892/1772] 10.222.0.52 () {58 vars in 1015 bytes} [Fri Aug 19 15:23:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:32.450908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:32.453048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:32.467927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065d2c3-5ca4-4179-8bcc-a16ab24c263e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:32.491308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065d2c3-5ca4-4179-8bcc-a16ab24c263e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:32.516248 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3e3d7-5468-4a65-8d2c-1f2a2662e1af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:32.517783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3e3d7-5468-4a65-8d2c-1f2a2662e1af service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:32.518545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2c034-cef2-4efd-9ecb-4c69c38ea5a9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:32.519814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3e3d7-5468-4a65-8d2c-1f2a2662e1af service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:32.520607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3e3d7-5468-4a65-8d2c-1f2a2662e1af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:32.538214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75ae0d-bbda-42f0-b3a7-df8b86af3bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:32.539362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75ae0d-bbda-42f0-b3a7-df8b86af3bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:32.541587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75ae0d-bbda-42f0-b3a7-df8b86af3bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:32.543158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75ae0d-bbda-42f0-b3a7-df8b86af3bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:32.555430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3e3d7-5468-4a65-8d2c-1f2a2662e1af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:32.564153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3e3d7-5468-4a65-8d2c-1f2a2662e1af service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:32.576098 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75ae0d-bbda-42f0-b3a7-df8b86af3bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:32.581550 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75ae0d-bbda-42f0-b3a7-df8b86af3bf8 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:32.598477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 893/1773] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:32.601922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3e3d7-5468-4a65-8d2c-1f2a2662e1af service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:32.614870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 881/1774] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:32.644721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3e3d7-5468-4a65-8d2c-1f2a2662e1af service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:32.655757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275e9f5-dec1-48bf-bc44-4c3a0b75b31a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:32.658570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275e9f5-dec1-48bf-bc44-4c3a0b75b31a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:32.660163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275e9f5-dec1-48bf-bc44-4c3a0b75b31a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:32.662873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275e9f5-dec1-48bf-bc44-4c3a0b75b31a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:32.711773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275e9f5-dec1-48bf-bc44-4c3a0b75b31a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:32.713464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275e9f5-dec1-48bf-bc44-4c3a0b75b31a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:32.749676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 894/1775] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:34.187166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75ae0d-bbda-42f0-b3a7-df8b86af3bf8 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:34.189515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5c75ae0d-bbda-42f0-b3a7-df8b86af3bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:34.320640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75ae0d-bbda-42f0-b3a7-df8b86af3bf8 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:34.331205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:34.332619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:34.333265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:34.333777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] PATH_INFO: `/v3/projects/56160baf00fb4e7c916b6ca6232f4fac` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:34.335687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56160baf00fb4e7c916b6ca6232f4fac)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:34.339219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:34.457354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56160baf00fb4e7c916b6ca6232f4fac', 'request_id': 'req-10fc29e5-7ea0-4d37-9cb7-983966f30338'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.481259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56160baf00fb4e7c916b6ca6232f4fac', 'request_id': 'req-10fc29e5-7ea0-4d37-9cb7-983966f30338'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.521141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56160baf00fb4e7c916b6ca6232f4fac', 'request_id': 'req-10fc29e5-7ea0-4d37-9cb7-983966f30338'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.550986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] The token cache is being invalidate because project 56160baf00fb4e7c916b6ca6232f4fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:34.551358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.552350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.555155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 882/1776] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:34 2022] DELETE /identity/v3/projects/56160baf00fb4e7c916b6ca6232f4fac => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:34.560977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275e9f5-dec1-48bf-bc44-4c3a0b75b31a service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:34.580244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275e9f5-dec1-48bf-bc44-4c3a0b75b31a service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:34.602442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:34.603915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:34.611138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:34.685521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:34.686286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:34.686666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:34.687175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] PATH_INFO: `/v3/users/7bc0fa0909c6484aa393c67f9d299805` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:34.689095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bc0fa0909c6484aa393c67f9d299805)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:34.691514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:34.815887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bc0fa0909c6484aa393c67f9d299805'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.816876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bc0fa0909c6484aa393c67f9d299805'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.817678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bc0fa0909c6484aa393c67f9d299805'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.832061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bc0fa0909c6484aa393c67f9d299805'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.874145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bc0fa0909c6484aa393c67f9d299805'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:34.879316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc29e5-7ea0-4d37-9cb7-983966f30338 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:34.899357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 895/1777] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:34 2022] DELETE /identity/v3/users/7bc0fa0909c6484aa393c67f9d299805 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:34.900518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:34.904658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:34.922442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:34.938448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:34.938897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:34.939304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] PATH_INFO: `/v3/projects/397314a16ec54459a03d5338cdda0d34` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:34.941815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=397314a16ec54459a03d5338cdda0d34)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:34.947209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:35.020702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:35.098349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d760c4-d241-4ebc-966b-0d1137567455 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:35.129990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '397314a16ec54459a03d5338cdda0d34', 'request_id': 'req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.145893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '397314a16ec54459a03d5338cdda0d34', 'request_id': 'req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.171084 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '397314a16ec54459a03d5338cdda0d34', 'request_id': 'req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.202843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] The token cache is being invalidate because project 397314a16ec54459a03d5338cdda0d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:35.203574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.210624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e7e0d2c8-8057-4c24-b8a2-5958e944ed0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.215514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 883/1778] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:34 2022] DELETE /identity/v3/projects/397314a16ec54459a03d5338cdda0d34 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:35.221002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efa246-6d3c-41ef-bee6-4c8157d19a9b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:35.226942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efa246-6d3c-41ef-bee6-4c8157d19a9b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:35.227357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efa246-6d3c-41ef-bee6-4c8157d19a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:35.301931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-52efa246-6d3c-41ef-bee6-4c8157d19a9b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:35.306517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-52efa246-6d3c-41ef-bee6-4c8157d19a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:35.337009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:35.338189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:35.339664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:35.340119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] PATH_INFO: `/v3/users/9aa29f1a9d044e92bbfdb2964a992c6d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:35.345193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa29f1a9d044e92bbfdb2964a992c6d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:35.348316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:35.470663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa29f1a9d044e92bbfdb2964a992c6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.473555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa29f1a9d044e92bbfdb2964a992c6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.475494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa29f1a9d044e92bbfdb2964a992c6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.491198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 884/1779] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:35.491587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa29f1a9d044e92bbfdb2964a992c6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.499620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a50d36-a6c9-486e-8ea9-ce9d0b926904 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:35.500045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a50d36-a6c9-486e-8ea9-ce9d0b926904 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:35.500526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a50d36-a6c9-486e-8ea9-ce9d0b926904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:35.552600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-74a50d36-a6c9-486e-8ea9-ce9d0b926904 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:35.554073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-74a50d36-a6c9-486e-8ea9-ce9d0b926904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:35.573986 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b0a4d299-dd59-4775-975a-f28bcf4b6ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa29f1a9d044e92bbfdb2964a992c6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:35.579710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 885/1780] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:35.584317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a50d36-a6c9-486e-8ea9-ce9d0b926904 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:35.599971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a50d36-a6c9-486e-8ea9-ce9d0b926904 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:35.615455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 896/1781] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:34 2022] DELETE /identity/v3/users/9aa29f1a9d044e92bbfdb2964a992c6d => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:35.623445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a566-f8dc-455e-8d32-d12ed8711b9c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:35.623902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a566-f8dc-455e-8d32-d12ed8711b9c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:35.624352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a566-f8dc-455e-8d32-d12ed8711b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:35.627569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922ea8f-006e-4f0e-908f-b9ad9fdf8d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:35.628450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922ea8f-006e-4f0e-908f-b9ad9fdf8d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:35.629280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922ea8f-006e-4f0e-908f-b9ad9fdf8d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:35.629908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922ea8f-006e-4f0e-908f-b9ad9fdf8d1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:35.630964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922ea8f-006e-4f0e-908f-b9ad9fdf8d1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:35.633446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922ea8f-006e-4f0e-908f-b9ad9fdf8d1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:35.637443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922ea8f-006e-4f0e-908f-b9ad9fdf8d1b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:35.675442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 886/1782] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:35.679307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6192a566-f8dc-455e-8d32-d12ed8711b9c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:35.682743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922ea8f-006e-4f0e-908f-b9ad9fdf8d1b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:35.683812 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6192a566-f8dc-455e-8d32-d12ed8711b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:35.718064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922ea8f-006e-4f0e-908f-b9ad9fdf8d1b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:35.740472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:35.834442 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 897/1783] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:35.843078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614d3fa-b2c8-42b2-a1fc-eb229720a3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:35.843837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614d3fa-b2c8-42b2-a1fc-eb229720a3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:35.844211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614d3fa-b2c8-42b2-a1fc-eb229720a3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:35.844579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614d3fa-b2c8-42b2-a1fc-eb229720a3b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:35.845344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614d3fa-b2c8-42b2-a1fc-eb229720a3b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:35.847475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614d3fa-b2c8-42b2-a1fc-eb229720a3b0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:35.887551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192a566-f8dc-455e-8d32-d12ed8711b9c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:35.890125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6192a566-f8dc-455e-8d32-d12ed8711b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:35.907926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 887/1784] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:35 2022] POST /identity/v3/projects => generated 373 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:35.911606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614d3fa-b2c8-42b2-a1fc-eb229720a3b0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:35.951619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:36.045504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192a566-f8dc-455e-8d32-d12ed8711b9c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:36.080016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614d3fa-b2c8-42b2-a1fc-eb229720a3b0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:36.094015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa365550-15e8-4fea-9bc9-4bd8e605d05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:36.094645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa365550-15e8-4fea-9bc9-4bd8e605d05c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:36.094912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa365550-15e8-4fea-9bc9-4bd8e605d05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:36.095193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa365550-15e8-4fea-9bc9-4bd8e605d05c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:36.095901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa365550-15e8-4fea-9bc9-4bd8e605d05c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:36.098298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa365550-15e8-4fea-9bc9-4bd8e605d05c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:36.107260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a5c0b-1d91-4678-9c4b-b7a3faaaa1f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:36.107950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5c0b-1d91-4678-9c4b-b7a3faaaa1f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:36.108265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5c0b-1d91-4678-9c4b-b7a3faaaa1f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:36.108588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a5c0b-1d91-4678-9c4b-b7a3faaaa1f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:36.123153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a5c0b-1d91-4678-9c4b-b7a3faaaa1f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:36.126112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a5c0b-1d91-4678-9c4b-b7a3faaaa1f2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:36.140924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 898/1785] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:36.145143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 888/1786] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:35 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:36.150551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a5c0b-1d91-4678-9c4b-b7a3faaaa1f2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:36.166328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a5c0b-1d91-4678-9c4b-b7a3faaaa1f2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:36.174177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd778-8bd8-48ef-9f1f-beebd25ac4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:36.174927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd778-8bd8-48ef-9f1f-beebd25ac4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:36.175398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd778-8bd8-48ef-9f1f-beebd25ac4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:36.175748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd778-8bd8-48ef-9f1f-beebd25ac4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:36.176497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acd778-8bd8-48ef-9f1f-beebd25ac4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:36.178433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acd778-8bd8-48ef-9f1f-beebd25ac4a6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:36.198884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 899/1787] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:36.200640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa365550-15e8-4fea-9bc9-4bd8e605d05c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:36.217522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa365550-15e8-4fea-9bc9-4bd8e605d05c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:36.237830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567599c-c131-494f-b67f-8b11e2d17d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:36.239099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567599c-c131-494f-b67f-8b11e2d17d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:36.239735 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567599c-c131-494f-b67f-8b11e2d17d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:36.240679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567599c-c131-494f-b67f-8b11e2d17d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:36.241880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567599c-c131-494f-b67f-8b11e2d17d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:36.246210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567599c-c131-494f-b67f-8b11e2d17d8d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:36.265986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 889/1788] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:23:36.270930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd778-8bd8-48ef-9f1f-beebd25ac4a6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:36.286479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd778-8bd8-48ef-9f1f-beebd25ac4a6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:36.295251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711819-0bef-43d2-9d9d-4ec725647a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:36.296152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711819-0bef-43d2-9d9d-4ec725647a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:36.296594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711819-0bef-43d2-9d9d-4ec725647a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:36.297027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711819-0bef-43d2-9d9d-4ec725647a3a admin admin] PATH_INFO: `/v3/projects/7e9cefbc1e18415492b1ea0801b92f8b/users/cab1d186ab1c43749b2e187bc19e4e79/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:36.318925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5711819-0bef-43d2-9d9d-4ec725647a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9cefbc1e18415492b1ea0801b92f8b, user_id=cab1d186ab1c43749b2e187bc19e4e79, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:36.322026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5711819-0bef-43d2-9d9d-4ec725647a3a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:36.348538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 900/1789] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:36 2022] PUT /identity/v3/projects/7e9cefbc1e18415492b1ea0801b92f8b/users/cab1d186ab1c43749b2e187bc19e4e79/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:36.363496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0efdea-f701-4a98-b6b5-df10abd0be47 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:36.365493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0efdea-f701-4a98-b6b5-df10abd0be47 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:36.366472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0efdea-f701-4a98-b6b5-df10abd0be47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:36.410127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bf0efdea-f701-4a98-b6b5-df10abd0be47 None None] MFA Rules not processed for user `cab1d186ab1c43749b2e187bc19e4e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:36.412444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bf0efdea-f701-4a98-b6b5-df10abd0be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:36.575782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711819-0bef-43d2-9d9d-4ec725647a3a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:36.576341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 890/1790] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:36.709868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711819-0bef-43d2-9d9d-4ec725647a3a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:36.718259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd16181-24a2-4148-9e30-203e48b40246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:36.719444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd16181-24a2-4148-9e30-203e48b40246 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:36.719764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd16181-24a2-4148-9e30-203e48b40246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:36.720093 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd16181-24a2-4148-9e30-203e48b40246 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:36.720931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd16181-24a2-4148-9e30-203e48b40246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:36.725664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd16181-24a2-4148-9e30-203e48b40246 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:36.757973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76080e-2110-448d-9922-149075eac0f2 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:36.758261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76080e-2110-448d-9922-149075eac0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:36.758524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76080e-2110-448d-9922-149075eac0f2 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:36.763243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 891/1791] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:23:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:23:36.779494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116c6c8-d60c-4306-aa07-a663bf861662 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:36.779997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116c6c8-d60c-4306-aa07-a663bf861662 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:36.780543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116c6c8-d60c-4306-aa07-a663bf861662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:36.816333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 901/1792] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:36 2022] POST /identity/v3/projects => generated 371 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:36.824423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd16181-24a2-4148-9e30-203e48b40246 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:36.896063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a116c6c8-d60c-4306-aa07-a663bf861662 None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:36.901560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a116c6c8-d60c-4306-aa07-a663bf861662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:36.922542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:36.993844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:37.104971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd16181-24a2-4148-9e30-203e48b40246 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:37.125390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe46a8-369c-41bb-b27c-1ef97de3a6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:37.126659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe46a8-369c-41bb-b27c-1ef97de3a6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:37.128689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe46a8-369c-41bb-b27c-1ef97de3a6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:37.129282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe46a8-369c-41bb-b27c-1ef97de3a6fb admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:37.131575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe46a8-369c-41bb-b27c-1ef97de3a6fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:37.137696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe46a8-369c-41bb-b27c-1ef97de3a6fb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:37.154444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 892/1793] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:23:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:37.193643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116c6c8-d60c-4306-aa07-a663bf861662 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:37.194724 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 902/1794] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:36 2022] POST /identity/v3/users => generated 292 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:37.212588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116c6c8-d60c-4306-aa07-a663bf861662 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:37.221087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf27dcb-7221-4cf9-976e-bcfc232683c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:37.222990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf27dcb-7221-4cf9-976e-bcfc232683c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:37.224298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf27dcb-7221-4cf9-976e-bcfc232683c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:37.225547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf27dcb-7221-4cf9-976e-bcfc232683c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:37.227459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf27dcb-7221-4cf9-976e-bcfc232683c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:37.232404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf27dcb-7221-4cf9-976e-bcfc232683c7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:37.253477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 893/1795] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:37.261544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe46a8-369c-41bb-b27c-1ef97de3a6fb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:37.288313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe46a8-369c-41bb-b27c-1ef97de3a6fb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:37.298527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cff91-71e8-4c50-a66d-56115e6f63c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:37.300520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cff91-71e8-4c50-a66d-56115e6f63c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:37.301120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cff91-71e8-4c50-a66d-56115e6f63c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:37.301611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cff91-71e8-4c50-a66d-56115e6f63c0 admin admin] PATH_INFO: `/v3/projects/3ec65f0f3fea4e498d7b4afda211efb1/users/3b12100b380b429e823ecf44f7f86ec0/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:37.339814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cff91-71e8-4c50-a66d-56115e6f63c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec65f0f3fea4e498d7b4afda211efb1, user_id=3b12100b380b429e823ecf44f7f86ec0, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:37.343883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cff91-71e8-4c50-a66d-56115e6f63c0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:37.369560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 903/1796] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:37 2022] PUT /identity/v3/projects/3ec65f0f3fea4e498d7b4afda211efb1/users/3b12100b380b429e823ecf44f7f86ec0/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:37.375432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf27dcb-7221-4cf9-976e-bcfc232683c7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:37.390802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf27dcb-7221-4cf9-976e-bcfc232683c7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:37.486971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cff91-71e8-4c50-a66d-56115e6f63c0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:37.490094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed5cff91-71e8-4c50-a66d-56115e6f63c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:37.524773 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca5d7e-82d6-4979-a0a1-5732f75929bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:37.525408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca5d7e-82d6-4979-a0a1-5732f75929bd admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:37.525716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca5d7e-82d6-4979-a0a1-5732f75929bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:37.525987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca5d7e-82d6-4979-a0a1-5732f75929bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:37.526677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca5d7e-82d6-4979-a0a1-5732f75929bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:37.528456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca5d7e-82d6-4979-a0a1-5732f75929bd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:37.554185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 894/1797] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:37.556671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca5d7e-82d6-4979-a0a1-5732f75929bd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:37.569555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca5d7e-82d6-4979-a0a1-5732f75929bd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:37.583266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f658c55-d653-4a25-ba85-4f473088d3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:37.584137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f658c55-d653-4a25-ba85-4f473088d3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:37.584301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f658c55-d653-4a25-ba85-4f473088d3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:37.584537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f658c55-d653-4a25-ba85-4f473088d3d0 admin admin] PATH_INFO: `/v3/projects/3ec65f0f3fea4e498d7b4afda211efb1/users/3b12100b380b429e823ecf44f7f86ec0/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:37.592654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f658c55-d653-4a25-ba85-4f473088d3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec65f0f3fea4e498d7b4afda211efb1, user_id=3b12100b380b429e823ecf44f7f86ec0, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:37.595707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f658c55-d653-4a25-ba85-4f473088d3d0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:37.631077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 895/1798] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:37 2022] PUT /identity/v3/projects/3ec65f0f3fea4e498d7b4afda211efb1/users/3b12100b380b429e823ecf44f7f86ec0/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:37.633743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e25e6-3ca8-4fcb-8328-8cb9a8fdb0c0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:37.634035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e25e6-3ca8-4fcb-8328-8cb9a8fdb0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:37.634368 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e25e6-3ca8-4fcb-8328-8cb9a8fdb0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:37.693580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cff91-71e8-4c50-a66d-56115e6f63c0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:37.712167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-105e25e6-3ca8-4fcb-8328-8cb9a8fdb0c0 None None] MFA Rules not processed for user `3b12100b380b429e823ecf44f7f86ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:37.714422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-105e25e6-3ca8-4fcb-8328-8cb9a8fdb0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:37.835024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44a876-f101-428c-b73f-060699156845 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:37.835838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44a876-f101-428c-b73f-060699156845 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:37.836289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44a876-f101-428c-b73f-060699156845 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:37.837005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44a876-f101-428c-b73f-060699156845 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:37.850661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44a876-f101-428c-b73f-060699156845 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:37.856529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44a876-f101-428c-b73f-060699156845 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:37.863304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:37.882409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 896/1799] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:37.882694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 904/1800] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:37.900108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e25e6-3ca8-4fcb-8328-8cb9a8fdb0c0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:37.902428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-105e25e6-3ca8-4fcb-8328-8cb9a8fdb0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:38.024117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44a876-f101-428c-b73f-060699156845 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:38.026806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8f44a876-f101-428c-b73f-060699156845 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:38.062567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e25e6-3ca8-4fcb-8328-8cb9a8fdb0c0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:38.070470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:38.073917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827411c-9af4-4194-8d71-75a0d40de7bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:38.075027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827411c-9af4-4194-8d71-75a0d40de7bd service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:38.075322 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827411c-9af4-4194-8d71-75a0d40de7bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:38.075600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827411c-9af4-4194-8d71-75a0d40de7bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:38.105095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827411c-9af4-4194-8d71-75a0d40de7bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:38.109048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827411c-9af4-4194-8d71-75a0d40de7bd service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:38.141291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 897/1801] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:38.234302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44a876-f101-428c-b73f-060699156845 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:38.250355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:38.251587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:38.252609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:38.252797 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] PATH_INFO: `/v3/users/99b2e057a01247d4960d1d7895bb465d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:38.262941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=99b2e057a01247d4960d1d7895bb465d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:38.269299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:38.302242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827411c-9af4-4194-8d71-75a0d40de7bd service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:38.312809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827411c-9af4-4194-8d71-75a0d40de7bd service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:38.323343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41dc82-e190-46a3-a4e0-ba936f38428e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:38.326544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41dc82-e190-46a3-a4e0-ba936f38428e service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:38.326839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41dc82-e190-46a3-a4e0-ba936f38428e service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:38.329077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41dc82-e190-46a3-a4e0-ba936f38428e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:38.429960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99b2e057a01247d4960d1d7895bb465d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:38.432813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99b2e057a01247d4960d1d7895bb465d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:38.435206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99b2e057a01247d4960d1d7895bb465d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:38.461260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99b2e057a01247d4960d1d7895bb465d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:38.514311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-faee14df-d912-4566-ade3-68bce1a00a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99b2e057a01247d4960d1d7895bb465d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:38.555297 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 905/1802] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:38 2022] DELETE /identity/v3/users/99b2e057a01247d4960d1d7895bb465d => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:38.559300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41dc82-e190-46a3-a4e0-ba936f38428e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:38.560012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a216c-257a-4387-af05-d6ef4ef09a3c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:38.560313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a216c-257a-4387-af05-d6ef4ef09a3c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:38.561594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a216c-257a-4387-af05-d6ef4ef09a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:38.563025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41dc82-e190-46a3-a4e0-ba936f38428e service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:38.614524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 898/1803] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:38.631404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a43a216c-257a-4387-af05-d6ef4ef09a3c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:38.632725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a43a216c-257a-4387-af05-d6ef4ef09a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:38.780513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 906/1804] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:38.954173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41dc82-e190-46a3-a4e0-ba936f38428e service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:38.956975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cb41dc82-e190-46a3-a4e0-ba936f38428e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:38.962514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a216c-257a-4387-af05-d6ef4ef09a3c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:38.964926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a43a216c-257a-4387-af05-d6ef4ef09a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:38.991490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41dc82-e190-46a3-a4e0-ba936f38428e service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:38.998600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:38.999494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:38.999920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:39.000354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] PATH_INFO: `/v3/projects/0c194bbe8007427cb9b45940d9e6ad87` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:39.003902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c194bbe8007427cb9b45940d9e6ad87)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:39.006624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:39.110414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:39.140054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c194bbe8007427cb9b45940d9e6ad87', 'request_id': 'req-f0f06409-47a6-43db-adda-5712e53d8127'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.158270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c194bbe8007427cb9b45940d9e6ad87', 'request_id': 'req-f0f06409-47a6-43db-adda-5712e53d8127'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.173929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a216c-257a-4387-af05-d6ef4ef09a3c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:39.176734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c194bbe8007427cb9b45940d9e6ad87', 'request_id': 'req-f0f06409-47a6-43db-adda-5712e53d8127'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.211625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] The token cache is being invalidate because project 0c194bbe8007427cb9b45940d9e6ad87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:39.211928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.219689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.225902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 899/1805] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:38 2022] DELETE /identity/v3/projects/0c194bbe8007427cb9b45940d9e6ad87 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:39.226894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:39.230727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:39.312227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:39.336795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:39.341199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:39.343132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:39.343618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:39.385355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:39.502961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:39.514171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:39.519061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f06409-47a6-43db-adda-5712e53d8127 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:39.530271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:39.532853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:39.533160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:39.533543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] PATH_INFO: `/v3/users/b9a347cfaf0443c3a4d27afc9666ffcf` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:39.534940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a347cfaf0443c3a4d27afc9666ffcf)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:39.538083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:39.546771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 907/1806] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:39.547861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:39.553186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:39.579449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ecda3-5225-4903-80ae-449cce9882db service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:39.593854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:39.594705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:39.595407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:39.595407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] PATH_INFO: `/v3/projects/55f1e21aa0c7480bab768655e10bb612` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:39.596720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f1e21aa0c7480bab768655e10bb612)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:39.599987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:39.675406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a347cfaf0443c3a4d27afc9666ffcf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.701242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a347cfaf0443c3a4d27afc9666ffcf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.702044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a347cfaf0443c3a4d27afc9666ffcf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.742676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a347cfaf0443c3a4d27afc9666ffcf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.755032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f1e21aa0c7480bab768655e10bb612', 'request_id': 'req-c30ab94c-bb39-4bef-b04c-56507a2434af'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.763020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bae5ff22-e79f-41ff-ad9c-94c5a0f33fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a347cfaf0443c3a4d27afc9666ffcf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.770231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f1e21aa0c7480bab768655e10bb612', 'request_id': 'req-c30ab94c-bb39-4bef-b04c-56507a2434af'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.775294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 900/1807] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:39 2022] DELETE /identity/v3/users/b9a347cfaf0443c3a4d27afc9666ffcf => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:39.786229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f1e21aa0c7480bab768655e10bb612', 'request_id': 'req-c30ab94c-bb39-4bef-b04c-56507a2434af'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.818792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] The token cache is being invalidate because project 55f1e21aa0c7480bab768655e10bb612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:39.819112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.821847 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c30ab94c-bb39-4bef-b04c-56507a2434af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:39.831296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 908/1808] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:39 2022] DELETE /identity/v3/projects/55f1e21aa0c7480bab768655e10bb612 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:39.833290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37426c8c-434f-4959-9f46-02f32bba0a41 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:39.833802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37426c8c-434f-4959-9f46-02f32bba0a41 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:39.834262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37426c8c-434f-4959-9f46-02f32bba0a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:39.876975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-37426c8c-434f-4959-9f46-02f32bba0a41 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:39.878486 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-37426c8c-434f-4959-9f46-02f32bba0a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:40.039308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960a512-c52d-4aa6-a6d0-33de10140038 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.039851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960a512-c52d-4aa6-a6d0-33de10140038 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.040936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960a512-c52d-4aa6-a6d0-33de10140038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:40.046660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 901/1809] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:40.084973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d960a512-c52d-4aa6-a6d0-33de10140038 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:40.093042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d960a512-c52d-4aa6-a6d0-33de10140038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:40.102039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37426c8c-434f-4959-9f46-02f32bba0a41 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:40.109930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 909/1810] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:40.118679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37426c8c-434f-4959-9f46-02f32bba0a41 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:40.133403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52037fd3-8886-45ce-a36c-7f8cf3e553ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:40.134422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52037fd3-8886-45ce-a36c-7f8cf3e553ab admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.136727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52037fd3-8886-45ce-a36c-7f8cf3e553ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.137531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52037fd3-8886-45ce-a36c-7f8cf3e553ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:40.138759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52037fd3-8886-45ce-a36c-7f8cf3e553ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:40.140717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52037fd3-8886-45ce-a36c-7f8cf3e553ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:40.143854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52037fd3-8886-45ce-a36c-7f8cf3e553ab admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:40.184635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960a512-c52d-4aa6-a6d0-33de10140038 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:40.198493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960a512-c52d-4aa6-a6d0-33de10140038 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:40.199789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 902/1811] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:40.219132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7065c5f-8099-41fb-b2f6-0f0cf6bcbe6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:40.222217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7065c5f-8099-41fb-b2f6-0f0cf6bcbe6c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.224292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7065c5f-8099-41fb-b2f6-0f0cf6bcbe6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.224771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7065c5f-8099-41fb-b2f6-0f0cf6bcbe6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:40.226081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7065c5f-8099-41fb-b2f6-0f0cf6bcbe6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:40.239093 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7065c5f-8099-41fb-b2f6-0f0cf6bcbe6c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:40.269608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:40.289417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 910/1812] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:40 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:40.289768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52037fd3-8886-45ce-a36c-7f8cf3e553ab admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:40.307129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52037fd3-8886-45ce-a36c-7f8cf3e553ab admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:40.381784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:40.470740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f593-8b09-4c23-8daa-591c7f44f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:40.471375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f593-8b09-4c23-8daa-591c7f44f585 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.471666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f593-8b09-4c23-8daa-591c7f44f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.471936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f593-8b09-4c23-8daa-591c7f44f585 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:40.472630 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f593-8b09-4c23-8daa-591c7f44f585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:40.474849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f593-8b09-4c23-8daa-591c7f44f585 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:40.521639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 903/1813] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:40 2022] POST /identity/v3/users => generated 294 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:40.525951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7065c5f-8099-41fb-b2f6-0f0cf6bcbe6c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:40.541091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7065c5f-8099-41fb-b2f6-0f0cf6bcbe6c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:40.548036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3230cc-6b9a-481d-b6ec-65d9aec0a9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:40.548888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3230cc-6b9a-481d-b6ec-65d9aec0a9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.549234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3230cc-6b9a-481d-b6ec-65d9aec0a9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.549574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3230cc-6b9a-481d-b6ec-65d9aec0a9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:40.550702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3230cc-6b9a-481d-b6ec-65d9aec0a9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:40.552715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3230cc-6b9a-481d-b6ec-65d9aec0a9f1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:40.575411 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 911/1814] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:40.578411 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f593-8b09-4c23-8daa-591c7f44f585 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:40.593604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f593-8b09-4c23-8daa-591c7f44f585 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:40.612774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c708138-3ef2-4767-b032-491dbbabcc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:40.613416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c708138-3ef2-4767-b032-491dbbabcc9e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.613706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c708138-3ef2-4767-b032-491dbbabcc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.613990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c708138-3ef2-4767-b032-491dbbabcc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:40.614655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c708138-3ef2-4767-b032-491dbbabcc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:40.616327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c708138-3ef2-4767-b032-491dbbabcc9e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:40.640862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 904/1815] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:40.645473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3230cc-6b9a-481d-b6ec-65d9aec0a9f1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:40.681143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3230cc-6b9a-481d-b6ec-65d9aec0a9f1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:40.700353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6382c3-0cf8-464d-9f03-64c51147ec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:40.701041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6382c3-0cf8-464d-9f03-64c51147ec02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.701326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6382c3-0cf8-464d-9f03-64c51147ec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.701607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6382c3-0cf8-464d-9f03-64c51147ec02 admin admin] PATH_INFO: `/v3/projects/f75d13b095e34c368cda8792ff7a553c/users/db64cfd432db40ce86d5e3cfd39f778d/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:40.740955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6382c3-0cf8-464d-9f03-64c51147ec02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75d13b095e34c368cda8792ff7a553c, user_id=db64cfd432db40ce86d5e3cfd39f778d, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:40.743721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6382c3-0cf8-464d-9f03-64c51147ec02 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:40.793364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 912/1816] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:40 2022] PUT /identity/v3/projects/f75d13b095e34c368cda8792ff7a553c/users/db64cfd432db40ce86d5e3cfd39f778d/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:40.795453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8bd03-dbbc-4985-97e7-ab1b2def0a38 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.795827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8bd03-dbbc-4985-97e7-ab1b2def0a38 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.796793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8bd03-dbbc-4985-97e7-ab1b2def0a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:40.865761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7ef8bd03-dbbc-4985-97e7-ab1b2def0a38 None None] MFA Rules not processed for user `db64cfd432db40ce86d5e3cfd39f778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:40.868113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7ef8bd03-dbbc-4985-97e7-ab1b2def0a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:40.941026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66628a5d-9041-4347-95e1-ce50a5db1cbd None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.941569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66628a5d-9041-4347-95e1-ce50a5db1cbd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.941816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66628a5d-9041-4347-95e1-ce50a5db1cbd None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:40.943820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 913/1817] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:23:40.980263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23467c61-fe62-4fa7-b82e-39756697abfb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:40.980667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23467c61-fe62-4fa7-b82e-39756697abfb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:40.980951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23467c61-fe62-4fa7-b82e-39756697abfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:41.016102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 905/1818] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:41.018170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8bd03-dbbc-4985-97e7-ab1b2def0a38 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:41.022119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-23467c61-fe62-4fa7-b82e-39756697abfb None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:41.023534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-23467c61-fe62-4fa7-b82e-39756697abfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:41.262274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 914/1819] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:23:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:41.271534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8bd03-dbbc-4985-97e7-ab1b2def0a38 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:41.288985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfc3bb-bd4f-4db8-87c2-5982e1656f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:41.289618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfc3bb-bd4f-4db8-87c2-5982e1656f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:41.289904 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfc3bb-bd4f-4db8-87c2-5982e1656f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:41.290184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfc3bb-bd4f-4db8-87c2-5982e1656f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:41.291101 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfc3bb-bd4f-4db8-87c2-5982e1656f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:41.295900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfc3bb-bd4f-4db8-87c2-5982e1656f60 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:41.408555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 906/1820] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:41 2022] POST /identity/v3/projects => generated 385 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:41.409233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23467c61-fe62-4fa7-b82e-39756697abfb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:41.426232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23467c61-fe62-4fa7-b82e-39756697abfb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:41.594416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fad0c8-9a1a-4c4c-a92c-69676ac249e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:41.595366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fad0c8-9a1a-4c4c-a92c-69676ac249e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:41.595766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fad0c8-9a1a-4c4c-a92c-69676ac249e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:41.596152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fad0c8-9a1a-4c4c-a92c-69676ac249e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:41.597216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fad0c8-9a1a-4c4c-a92c-69676ac249e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:41.603072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfc3bb-bd4f-4db8-87c2-5982e1656f60 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:41.604068 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fad0c8-9a1a-4c4c-a92c-69676ac249e5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:41.610592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-90dfc3bb-bd4f-4db8-87c2-5982e1656f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:41.665902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 915/1821] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:41 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:23:41.670986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfc3bb-bd4f-4db8-87c2-5982e1656f60 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:41.675118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fad0c8-9a1a-4c4c-a92c-69676ac249e5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:41.684297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:41.685379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:41.685584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:41.685861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] PATH_INFO: `/v3/projects/2159758dea7c4b23a31fa18fde2fb863` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:41.693713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2159758dea7c4b23a31fa18fde2fb863)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:41.696444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:41.719399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fad0c8-9a1a-4c4c-a92c-69676ac249e5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:41.730958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:41.735491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f4e51-0de9-4a63-9827-c6f58d06df17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:41.737478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f4e51-0de9-4a63-9827-c6f58d06df17 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:41.737765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f4e51-0de9-4a63-9827-c6f58d06df17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:41.738044 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f4e51-0de9-4a63-9827-c6f58d06df17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:41.738936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f4e51-0de9-4a63-9827-c6f58d06df17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:41.742858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f4e51-0de9-4a63-9827-c6f58d06df17 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:41.782272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 916/1822] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:41.792569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f4e51-0de9-4a63-9827-c6f58d06df17 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:41.818277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2159758dea7c4b23a31fa18fde2fb863', 'request_id': 'req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:41.820306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f4e51-0de9-4a63-9827-c6f58d06df17 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:41.835101 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2159758dea7c4b23a31fa18fde2fb863', 'request_id': 'req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:41.857480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2159758dea7c4b23a31fa18fde2fb863', 'request_id': 'req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:41.903469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:41.941855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] The token cache is being invalidate because project 2159758dea7c4b23a31fa18fde2fb863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:41.945659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:41.958750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:41.969531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 907/1823] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:41 2022] DELETE /identity/v3/projects/2159758dea7c4b23a31fa18fde2fb863 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:41.972027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:41.973318 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319b927-f814-476c-accd-fadc92ee3de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:41.973943 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b927-f814-476c-accd-fadc92ee3de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:41.974229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b927-f814-476c-accd-fadc92ee3de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:41.974507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319b927-f814-476c-accd-fadc92ee3de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:41.975171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319b927-f814-476c-accd-fadc92ee3de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:41.977138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:41.981695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0319b927-f814-476c-accd-fadc92ee3de1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:42.018710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7b1f2-4fbf-443c-ab68-8b05c4821915 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:42.036493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 917/1824] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:42.038309 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:42.039072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:42.039227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:42.040733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319b927-f814-476c-accd-fadc92ee3de1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:42.046356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] PATH_INFO: `/v3/users/5eb019fe2e374a3aa07f67aaad22ee05` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:42.047092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0319b927-f814-476c-accd-fadc92ee3de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:42.048168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eb019fe2e374a3aa07f67aaad22ee05)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:42.063926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:42.085025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319b927-f814-476c-accd-fadc92ee3de1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:42.101494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db852e-b312-4be2-ad8d-cef2a116d00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:42.102312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db852e-b312-4be2-ad8d-cef2a116d00a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:42.102646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db852e-b312-4be2-ad8d-cef2a116d00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:42.102967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db852e-b312-4be2-ad8d-cef2a116d00a admin admin] PATH_INFO: `/v3/projects/0956cca1f69f4103a1e7f2aad89a23e2/users/1158e4b2c6ff4009bbc5e85edba7d986/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:42.149343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db852e-b312-4be2-ad8d-cef2a116d00a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0956cca1f69f4103a1e7f2aad89a23e2, user_id=1158e4b2c6ff4009bbc5e85edba7d986, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:42.153085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db852e-b312-4be2-ad8d-cef2a116d00a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:42.182399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 918/1825] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:42 2022] PUT /identity/v3/projects/0956cca1f69f4103a1e7f2aad89a23e2/users/1158e4b2c6ff4009bbc5e85edba7d986/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:42.195831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db852e-b312-4be2-ad8d-cef2a116d00a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:42.195831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e2db852e-b312-4be2-ad8d-cef2a116d00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:42.270813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eb019fe2e374a3aa07f67aaad22ee05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:42.282834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eb019fe2e374a3aa07f67aaad22ee05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:42.283227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eb019fe2e374a3aa07f67aaad22ee05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:42.309837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eb019fe2e374a3aa07f67aaad22ee05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:42.324513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-990ac51f-d346-4d38-bdf6-b04d78ebeaa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eb019fe2e374a3aa07f67aaad22ee05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:42.352366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 908/1826] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:41 2022] DELETE /identity/v3/users/5eb019fe2e374a3aa07f67aaad22ee05 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:42.357353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25c000-311e-4a07-8a53-d3e98ad5fb26 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:42.358325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25c000-311e-4a07-8a53-d3e98ad5fb26 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:42.358677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25c000-311e-4a07-8a53-d3e98ad5fb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:42.402659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db852e-b312-4be2-ad8d-cef2a116d00a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:42.434270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3e25c000-311e-4a07-8a53-d3e98ad5fb26 None None] MFA Rules not processed for user `1158e4b2c6ff4009bbc5e85edba7d986`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:42.435832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3e25c000-311e-4a07-8a53-d3e98ad5fb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:42.564626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:42.565239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:42.565580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:42.565860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:42.567920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:42.596962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:42.599063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:42.630259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 919/1827] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:42.631129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:42.636586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:42.710193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 909/1828] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:42.715866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25c000-311e-4a07-8a53-d3e98ad5fb26 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:42.718423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3e25c000-311e-4a07-8a53-d3e98ad5fb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:42.725657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:42.740111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:42.817306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ebd64-3868-4d09-8a7f-4d1a6a4b038d service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:42.831196 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:42.831919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:42.832235 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:42.832539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] PATH_INFO: `/v3/projects/57bf2744971d4d67b602bf5798fe9627` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:42.833802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=57bf2744971d4d67b602bf5798fe9627)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:42.835998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:42.942593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57bf2744971d4d67b602bf5798fe9627', 'request_id': 'req-999b6a5b-333f-479a-80d9-c153daebb34b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:42.958039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57bf2744971d4d67b602bf5798fe9627', 'request_id': 'req-999b6a5b-333f-479a-80d9-c153daebb34b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:42.975786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57bf2744971d4d67b602bf5798fe9627', 'request_id': 'req-999b6a5b-333f-479a-80d9-c153daebb34b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:42.999692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] The token cache is being invalidate because project 57bf2744971d4d67b602bf5798fe9627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:43.000036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:43.001918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-999b6a5b-333f-479a-80d9-c153daebb34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:43.006934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 920/1829] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:42 2022] DELETE /identity/v3/projects/57bf2744971d4d67b602bf5798fe9627 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:43.011879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae2b20-347c-45fc-8d8d-4b8dccab43bd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.012176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25c000-311e-4a07-8a53-d3e98ad5fb26 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.012281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae2b20-347c-45fc-8d8d-4b8dccab43bd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.012521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae2b20-347c-45fc-8d8d-4b8dccab43bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.055436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-49ae2b20-347c-45fc-8d8d-4b8dccab43bd None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:43.060929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49ae2b20-347c-45fc-8d8d-4b8dccab43bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:43.116338 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295317-0c36-4d0e-8d9f-e1dcf74b9fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.117472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295317-0c36-4d0e-8d9f-e1dcf74b9fcb admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.117957 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295317-0c36-4d0e-8d9f-e1dcf74b9fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.118446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295317-0c36-4d0e-8d9f-e1dcf74b9fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.119177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf295317-0c36-4d0e-8d9f-e1dcf74b9fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.122249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf295317-0c36-4d0e-8d9f-e1dcf74b9fcb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.170846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 910/1830] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:42 2022] POST /identity/v3/projects => generated 385 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:43.173359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295317-0c36-4d0e-8d9f-e1dcf74b9fcb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.175453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cf295317-0c36-4d0e-8d9f-e1dcf74b9fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:43.207319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 921/1831] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:43.217833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295ab55-fa67-4228-9098-69fab5ad08f7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.218457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295ab55-fa67-4228-9098-69fab5ad08f7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.219423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295ab55-fa67-4228-9098-69fab5ad08f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.255217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a295ab55-fa67-4228-9098-69fab5ad08f7 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:43.256565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a295ab55-fa67-4228-9098-69fab5ad08f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:43.281163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295317-0c36-4d0e-8d9f-e1dcf74b9fcb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.308089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683a6b2-2276-4d83-b27d-f03adf250554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.308969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683a6b2-2276-4d83-b27d-f03adf250554 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.309428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683a6b2-2276-4d83-b27d-f03adf250554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.309845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683a6b2-2276-4d83-b27d-f03adf250554 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.311108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683a6b2-2276-4d83-b27d-f03adf250554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.316834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683a6b2-2276-4d83-b27d-f03adf250554 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.323660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:43.335097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 922/1832] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:43.343854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295ab55-fa67-4228-9098-69fab5ad08f7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.358082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295ab55-fa67-4228-9098-69fab5ad08f7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.363842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d64d4e-1beb-4de3-a716-08a187e07e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.364584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d64d4e-1beb-4de3-a716-08a187e07e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.364861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d64d4e-1beb-4de3-a716-08a187e07e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.365136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d64d4e-1beb-4de3-a716-08a187e07e5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.365794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d64d4e-1beb-4de3-a716-08a187e07e5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:43.368832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d64d4e-1beb-4de3-a716-08a187e07e5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.368832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d64d4e-1beb-4de3-a716-08a187e07e5e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.369417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 911/1833] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:43 2022] POST /identity/v3/users => generated 299 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:43.371259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683a6b2-2276-4d83-b27d-f03adf250554 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.390084 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683a6b2-2276-4d83-b27d-f03adf250554 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.396082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf5d96-bf9d-4347-8b1e-d6bde43fef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.397441 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf5d96-bf9d-4347-8b1e-d6bde43fef18 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.397996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf5d96-bf9d-4347-8b1e-d6bde43fef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.398323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf5d96-bf9d-4347-8b1e-d6bde43fef18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.399165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf5d96-bf9d-4347-8b1e-d6bde43fef18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.402491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf5d96-bf9d-4347-8b1e-d6bde43fef18 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.408725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 923/1834] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:23:43.413285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d64d4e-1beb-4de3-a716-08a187e07e5e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.425851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 912/1835] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:43.428458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d64d4e-1beb-4de3-a716-08a187e07e5e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.430501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf5d96-bf9d-4347-8b1e-d6bde43fef18 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.434968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cdead-7ef3-4a96-8212-13fa66abc34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.435822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cdead-7ef3-4a96-8212-13fa66abc34c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.436205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cdead-7ef3-4a96-8212-13fa66abc34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.436564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cdead-7ef3-4a96-8212-13fa66abc34c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.437306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8cdead-7ef3-4a96-8212-13fa66abc34c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.439481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8cdead-7ef3-4a96-8212-13fa66abc34c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.442333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf5d96-bf9d-4347-8b1e-d6bde43fef18 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.453223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db647ec-c4c0-4cee-9f13-2515f270e910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.454210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db647ec-c4c0-4cee-9f13-2515f270e910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.454597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db647ec-c4c0-4cee-9f13-2515f270e910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.454976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db647ec-c4c0-4cee-9f13-2515f270e910 admin admin] PATH_INFO: `/v3/projects/2cddbe7eb64943e4bc27b926e198c828/users/faf1d62615fd466c800a0b54e2214d4a/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.477385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db647ec-c4c0-4cee-9f13-2515f270e910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cddbe7eb64943e4bc27b926e198c828, user_id=faf1d62615fd466c800a0b54e2214d4a, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.478361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 924/1836] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:43 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:23:43.481212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db647ec-c4c0-4cee-9f13-2515f270e910 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.483150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cdead-7ef3-4a96-8212-13fa66abc34c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.484037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:43.504378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 913/1837] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:43 2022] PUT /identity/v3/projects/2cddbe7eb64943e4bc27b926e198c828/users/faf1d62615fd466c800a0b54e2214d4a/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:43.509665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db647ec-c4c0-4cee-9f13-2515f270e910 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.613132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cdead-7ef3-4a96-8212-13fa66abc34c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.632903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db647ec-c4c0-4cee-9f13-2515f270e910 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.640220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4188cc-dedf-4c95-8e10-f40a8bf2c4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.640948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4188cc-dedf-4c95-8e10-f40a8bf2c4ad admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.641695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4188cc-dedf-4c95-8e10-f40a8bf2c4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.642070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4188cc-dedf-4c95-8e10-f40a8bf2c4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.642834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4188cc-dedf-4c95-8e10-f40a8bf2c4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.644635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4188cc-dedf-4c95-8e10-f40a8bf2c4ad admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.657502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 914/1838] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:43.661877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4188cc-dedf-4c95-8e10-f40a8bf2c4ad admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.674591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4188cc-dedf-4c95-8e10-f40a8bf2c4ad admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.691703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3da65-a3a5-439f-95cc-d88bc4a29f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.692665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3da65-a3a5-439f-95cc-d88bc4a29f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.693080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3da65-a3a5-439f-95cc-d88bc4a29f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.693594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3da65-a3a5-439f-95cc-d88bc4a29f33 admin admin] PATH_INFO: `/v3/projects/2cddbe7eb64943e4bc27b926e198c828/users/faf1d62615fd466c800a0b54e2214d4a/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.696780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3da65-a3a5-439f-95cc-d88bc4a29f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cddbe7eb64943e4bc27b926e198c828, user_id=faf1d62615fd466c800a0b54e2214d4a, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.710666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3da65-a3a5-439f-95cc-d88bc4a29f33 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.728601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e635b09-e95f-48e0-a9f2-200e37d8004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.730026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e635b09-e95f-48e0-a9f2-200e37d8004b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.730520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e635b09-e95f-48e0-a9f2-200e37d8004b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.730842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e635b09-e95f-48e0-a9f2-200e37d8004b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.731497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e635b09-e95f-48e0-a9f2-200e37d8004b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.733980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e635b09-e95f-48e0-a9f2-200e37d8004b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.736555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 915/1839] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:43 2022] PUT /identity/v3/projects/2cddbe7eb64943e4bc27b926e198c828/users/faf1d62615fd466c800a0b54e2214d4a/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:43.741554 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3da65-a3a5-439f-95cc-d88bc4a29f33 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.749531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-61f3da65-a3a5-439f-95cc-d88bc4a29f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:43.763090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 925/1840] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:43 2022] POST /identity/v3/users => generated 294 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:23:43.764694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4bad7-32a1-4589-b4c2-2685bcff1e04 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.765141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4bad7-32a1-4589-b4c2-2685bcff1e04 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.765453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4bad7-32a1-4589-b4c2-2685bcff1e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.802945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6cc4bad7-32a1-4589-b4c2-2685bcff1e04 None None] MFA Rules not processed for user `faf1d62615fd466c800a0b54e2214d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:43.804224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6cc4bad7-32a1-4589-b4c2-2685bcff1e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:43.857182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3da65-a3a5-439f-95cc-d88bc4a29f33 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.863290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:43.863977 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:43.864262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:43.864588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] PATH_INFO: `/v3/projects/c323083ac58246eb9339b61d7eb4bb80` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:43.865791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c323083ac58246eb9339b61d7eb4bb80)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:43.868041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:43.893128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:43.927190 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 926/1841] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:43.927190 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4bad7-32a1-4589-b4c2-2685bcff1e04 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.936212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c323083ac58246eb9339b61d7eb4bb80', 'request_id': 'req-3f15d50f-d072-4249-b982-591d432923c4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:43.939480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4bad7-32a1-4589-b4c2-2685bcff1e04 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:43.943018 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c323083ac58246eb9339b61d7eb4bb80', 'request_id': 'req-3f15d50f-d072-4249-b982-591d432923c4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:43.951841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c323083ac58246eb9339b61d7eb4bb80', 'request_id': 'req-3f15d50f-d072-4249-b982-591d432923c4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:43.965370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] The token cache is being invalidate because project c323083ac58246eb9339b61d7eb4bb80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:43.965641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:43.966318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:43.967821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 916/1842] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:43 2022] DELETE /identity/v3/projects/c323083ac58246eb9339b61d7eb4bb80 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:43.968628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:43.970061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:43.981678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:44.014579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b4c2c-b633-468b-8a22-70d7abef7bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.015505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b4c2c-b633-468b-8a22-70d7abef7bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.015938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b4c2c-b633-468b-8a22-70d7abef7bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.016487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b4c2c-b633-468b-8a22-70d7abef7bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.018390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b4c2c-b633-468b-8a22-70d7abef7bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.020027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b4c2c-b633-468b-8a22-70d7abef7bbd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.031942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 927/1843] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:44.035013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897968d-7567-4d44-b7ea-14b961c002b8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.035617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897968d-7567-4d44-b7ea-14b961c002b8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.036226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897968d-7567-4d44-b7ea-14b961c002b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.056940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15d50f-d072-4249-b982-591d432923c4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.058687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6897968d-7567-4d44-b7ea-14b961c002b8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:44.060132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6897968d-7567-4d44-b7ea-14b961c002b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:44.061069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d49c9-699d-4a13-9b5c-b1a11f3781be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.061694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d49c9-699d-4a13-9b5c-b1a11f3781be service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.061979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d49c9-699d-4a13-9b5c-b1a11f3781be service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.062332 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d49c9-699d-4a13-9b5c-b1a11f3781be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.064191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-240d49c9-699d-4a13-9b5c-b1a11f3781be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:44.069741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 928/1844] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:44.070519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897968d-7567-4d44-b7ea-14b961c002b8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.071951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6897968d-7567-4d44-b7ea-14b961c002b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:44.085451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897968d-7567-4d44-b7ea-14b961c002b8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.092712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17207dc3-7010-4fae-9be8-6cd70dd83a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.093428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17207dc3-7010-4fae-9be8-6cd70dd83a08 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.093715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17207dc3-7010-4fae-9be8-6cd70dd83a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.093994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17207dc3-7010-4fae-9be8-6cd70dd83a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.094750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17207dc3-7010-4fae-9be8-6cd70dd83a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.096734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17207dc3-7010-4fae-9be8-6cd70dd83a08 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.109294 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 929/1845] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:44.110723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dfbbd-2622-4a08-b765-78529027a285 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.111092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dfbbd-2622-4a08-b765-78529027a285 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.111377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dfbbd-2622-4a08-b765-78529027a285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.136998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3a5dfbbd-2622-4a08-b765-78529027a285 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:44.138444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3a5dfbbd-2622-4a08-b765-78529027a285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:44.147231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d49c9-699d-4a13-9b5c-b1a11f3781be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.148382 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 930/1846] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:44.148945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d49c9-699d-4a13-9b5c-b1a11f3781be service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.149527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dfbbd-2622-4a08-b765-78529027a285 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.160086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 917/1847] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:44.161001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d49c9-699d-4a13-9b5c-b1a11f3781be service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.164480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dfbbd-2622-4a08-b765-78529027a285 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.171900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0471d4-853b-492f-be74-75eb181e885a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.172033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d49c9-699d-4a13-9b5c-b1a11f3781be service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.172885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0471d4-853b-492f-be74-75eb181e885a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.173277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0471d4-853b-492f-be74-75eb181e885a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.173641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0471d4-853b-492f-be74-75eb181e885a admin admin] PATH_INFO: `/v3/projects/82e150d42bb44293b184c3d5c5fd2dee/users/9894395bab9c4e838db8185069be2def/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.176785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3931d8d-390d-4ce4-8a15-d447da7789da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.177404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3931d8d-390d-4ce4-8a15-d447da7789da admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.177690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3931d8d-390d-4ce4-8a15-d447da7789da admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.177969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3931d8d-390d-4ce4-8a15-d447da7789da admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.178672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3931d8d-390d-4ce4-8a15-d447da7789da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:44.178983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3931d8d-390d-4ce4-8a15-d447da7789da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.180630 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3931d8d-390d-4ce4-8a15-d447da7789da admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.194585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0471d4-853b-492f-be74-75eb181e885a admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e150d42bb44293b184c3d5c5fd2dee, user_id=9894395bab9c4e838db8185069be2def, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.195262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 918/1848] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:44.197356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0471d4-853b-492f-be74-75eb181e885a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.199707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3931d8d-390d-4ce4-8a15-d447da7789da admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.212464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3931d8d-390d-4ce4-8a15-d447da7789da admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.215344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 931/1849] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:44 2022] PUT /identity/v3/projects/82e150d42bb44293b184c3d5c5fd2dee/users/9894395bab9c4e838db8185069be2def/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:44.225645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb4c9c-b71a-4ed7-8e5a-d168723f300a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.225935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb4c9c-b71a-4ed7-8e5a-d168723f300a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.226270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb4c9c-b71a-4ed7-8e5a-d168723f300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.276652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7bcb4c9c-b71a-4ed7-8e5a-d168723f300a None None] MFA Rules not processed for user `9894395bab9c4e838db8185069be2def`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:44.278512 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7bcb4c9c-b71a-4ed7-8e5a-d168723f300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:44.302567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d80fa-7813-4959-a695-34a7cc984e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.303377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d80fa-7813-4959-a695-34a7cc984e11 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.303858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d80fa-7813-4959-a695-34a7cc984e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.304151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d80fa-7813-4959-a695-34a7cc984e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.304955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d80fa-7813-4959-a695-34a7cc984e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.307034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d80fa-7813-4959-a695-34a7cc984e11 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.377047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 919/1850] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:44 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:23:44.378277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d80fa-7813-4959-a695-34a7cc984e11 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.389445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 932/1851] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:44.390429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb4c9c-b71a-4ed7-8e5a-d168723f300a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.414383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb4c9c-b71a-4ed7-8e5a-d168723f300a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.440997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:44.455464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:44.509469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d80fa-7813-4959-a695-34a7cc984e11 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.518395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b860e-84b8-4edf-8623-d5c6ebaff8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.519362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b860e-84b8-4edf-8623-d5c6ebaff8bd admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.519737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b860e-84b8-4edf-8623-d5c6ebaff8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.520110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b860e-84b8-4edf-8623-d5c6ebaff8bd admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.521654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b860e-84b8-4edf-8623-d5c6ebaff8bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.524545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b860e-84b8-4edf-8623-d5c6ebaff8bd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.536065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c5b10-bd9c-4663-861e-2c2b7dd1ede2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.544602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c5b10-bd9c-4663-861e-2c2b7dd1ede2 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.544602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c5b10-bd9c-4663-861e-2c2b7dd1ede2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.544602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c5b10-bd9c-4663-861e-2c2b7dd1ede2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.544602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c5b10-bd9c-4663-861e-2c2b7dd1ede2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.546593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c5b10-bd9c-4663-861e-2c2b7dd1ede2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.576592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 920/1852] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:44 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:23:44.579286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b860e-84b8-4edf-8623-d5c6ebaff8bd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.601184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b860e-84b8-4edf-8623-d5c6ebaff8bd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.610481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb1df4-7772-41bf-a6e4-3ebb711edce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.611491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb1df4-7772-41bf-a6e4-3ebb711edce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.612055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb1df4-7772-41bf-a6e4-3ebb711edce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.612670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb1df4-7772-41bf-a6e4-3ebb711edce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.613859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb1df4-7772-41bf-a6e4-3ebb711edce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.617299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb1df4-7772-41bf-a6e4-3ebb711edce9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.632226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 933/1853] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:44 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:44.633658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c5b10-bd9c-4663-861e-2c2b7dd1ede2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.663941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 921/1854] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:44.665989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb1df4-7772-41bf-a6e4-3ebb711edce9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.686590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb1df4-7772-41bf-a6e4-3ebb711edce9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.759548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c5b10-bd9c-4663-861e-2c2b7dd1ede2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.768985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed782300-6ff0-43df-a1dc-1f533d2427d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.779179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed782300-6ff0-43df-a1dc-1f533d2427d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.779674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed782300-6ff0-43df-a1dc-1f533d2427d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.780027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed782300-6ff0-43df-a1dc-1f533d2427d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.780989 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed782300-6ff0-43df-a1dc-1f533d2427d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.783507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed782300-6ff0-43df-a1dc-1f533d2427d4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.787098 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dee5f9-b616-4714-abe3-f81f7d5da1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.787728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dee5f9-b616-4714-abe3-f81f7d5da1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.788018 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dee5f9-b616-4714-abe3-f81f7d5da1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.788292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dee5f9-b616-4714-abe3-f81f7d5da1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.789010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dee5f9-b616-4714-abe3-f81f7d5da1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.791039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dee5f9-b616-4714-abe3-f81f7d5da1e8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.814289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 922/1855] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:44.814547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 934/1856] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:44 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:44.821321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed782300-6ff0-43df-a1dc-1f533d2427d4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.821553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dee5f9-b616-4714-abe3-f81f7d5da1e8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.832796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed782300-6ff0-43df-a1dc-1f533d2427d4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.834596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dee5f9-b616-4714-abe3-f81f7d5da1e8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.839939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11908c29-3f44-4369-bfb1-2a2f5059ae78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.840459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11908c29-3f44-4369-bfb1-2a2f5059ae78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.840735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11908c29-3f44-4369-bfb1-2a2f5059ae78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.841015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11908c29-3f44-4369-bfb1-2a2f5059ae78 admin admin] PATH_INFO: `/v3/projects/1a8aace3fc7348d2877c1b96e1df9bf4/users/79467cf25fa34db08a8ba53afedc75fd/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.842609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91b402-82c3-4e2d-85d9-50a693d446d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.843511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91b402-82c3-4e2d-85d9-50a693d446d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.843934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91b402-82c3-4e2d-85d9-50a693d446d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.844357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91b402-82c3-4e2d-85d9-50a693d446d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.845320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91b402-82c3-4e2d-85d9-50a693d446d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.847202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91b402-82c3-4e2d-85d9-50a693d446d9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.876902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11908c29-3f44-4369-bfb1-2a2f5059ae78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8aace3fc7348d2877c1b96e1df9bf4, user_id=79467cf25fa34db08a8ba53afedc75fd, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.878767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 923/1857] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:44.879507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91b402-82c3-4e2d-85d9-50a693d446d9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:44.879763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11908c29-3f44-4369-bfb1-2a2f5059ae78 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.896266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91b402-82c3-4e2d-85d9-50a693d446d9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:44.904782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cbb68-2765-4368-b904-59d6dfc59e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:44.905593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cbb68-2765-4368-b904-59d6dfc59e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.912971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 935/1858] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:44 2022] PUT /identity/v3/projects/1a8aace3fc7348d2877c1b96e1df9bf4/users/79467cf25fa34db08a8ba53afedc75fd/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:44.917792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cbb68-2765-4368-b904-59d6dfc59e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.917792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cbb68-2765-4368-b904-59d6dfc59e6f admin admin] PATH_INFO: `/v3/projects/780ed9721711450b98b8b32f3018cb53/users/3dc05026199e43368fe6d333215b0cec/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.931623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a282e49-fae7-4b94-8720-1c7d51325059 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:44.932957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a282e49-fae7-4b94-8720-1c7d51325059 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:44.933421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a282e49-fae7-4b94-8720-1c7d51325059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:44.951988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cbb68-2765-4368-b904-59d6dfc59e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=780ed9721711450b98b8b32f3018cb53, user_id=3dc05026199e43368fe6d333215b0cec, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:44.954975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cbb68-2765-4368-b904-59d6dfc59e6f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:44.974463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 924/1859] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:44 2022] PUT /identity/v3/projects/780ed9721711450b98b8b32f3018cb53/users/3dc05026199e43368fe6d333215b0cec/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:44.974866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8a282e49-fae7-4b94-8720-1c7d51325059 None None] MFA Rules not processed for user `79467cf25fa34db08a8ba53afedc75fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:44.976166 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a282e49-fae7-4b94-8720-1c7d51325059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:44.978619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cbb68-2765-4368-b904-59d6dfc59e6f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.070289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cbb68-2765-4368-b904-59d6dfc59e6f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.077903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6388-6bcb-4409-b571-c49526b1d942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:45.078523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6388-6bcb-4409-b571-c49526b1d942 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.078806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6388-6bcb-4409-b571-c49526b1d942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.079077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea6388-6bcb-4409-b571-c49526b1d942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.079731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6388-6bcb-4409-b571-c49526b1d942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:45.081479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea6388-6bcb-4409-b571-c49526b1d942 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:45.092501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 936/1860] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:45.097646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a282e49-fae7-4b94-8720-1c7d51325059 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.107023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 925/1861] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:23:45.112137 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:45.113166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6388-6bcb-4409-b571-c49526b1d942 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.121410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a282e49-fae7-4b94-8720-1c7d51325059 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.124010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:45.127886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf19c57-8b22-4f98-b0df-11ed7755ac99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:45.128533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf19c57-8b22-4f98-b0df-11ed7755ac99 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.128814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf19c57-8b22-4f98-b0df-11ed7755ac99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.129084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf19c57-8b22-4f98-b0df-11ed7755ac99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.129939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf19c57-8b22-4f98-b0df-11ed7755ac99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:45.132117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf19c57-8b22-4f98-b0df-11ed7755ac99 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:45.132260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea6388-6bcb-4409-b571-c49526b1d942 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.140914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc2f0c-1433-4f87-bfa9-0267feae6bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:45.141723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc2f0c-1433-4f87-bfa9-0267feae6bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.142015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc2f0c-1433-4f87-bfa9-0267feae6bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.142296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc2f0c-1433-4f87-bfa9-0267feae6bcc admin admin] PATH_INFO: `/v3/projects/780ed9721711450b98b8b32f3018cb53/users/3dc05026199e43368fe6d333215b0cec/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.144542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc2f0c-1433-4f87-bfa9-0267feae6bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=780ed9721711450b98b8b32f3018cb53, user_id=3dc05026199e43368fe6d333215b0cec, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:45.147610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc2f0c-1433-4f87-bfa9-0267feae6bcc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:45.167145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 937/1863] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:45 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:45.168174 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf19c57-8b22-4f98-b0df-11ed7755ac99 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.170111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8bf19c57-8b22-4f98-b0df-11ed7755ac99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:45.172466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 926/1863] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:45 2022] PUT /identity/v3/projects/780ed9721711450b98b8b32f3018cb53/users/3dc05026199e43368fe6d333215b0cec/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:45.178277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc2f0c-1433-4f87-bfa9-0267feae6bcc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.248047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf19c57-8b22-4f98-b0df-11ed7755ac99 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.252239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24d89-6c18-4b35-be3a-766a718f9d1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:45.253219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24d89-6c18-4b35-be3a-766a718f9d1a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.253509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24d89-6c18-4b35-be3a-766a718f9d1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.254016 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24d89-6c18-4b35-be3a-766a718f9d1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.256907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-47b24d89-6c18-4b35-be3a-766a718f9d1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:45.271163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc2f0c-1433-4f87-bfa9-0267feae6bcc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.333320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a52c2f-7680-42ff-80fd-a333f7b4a1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:45.334052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a52c2f-7680-42ff-80fd-a333f7b4a1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.334328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a52c2f-7680-42ff-80fd-a333f7b4a1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.334599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a52c2f-7680-42ff-80fd-a333f7b4a1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.335268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a52c2f-7680-42ff-80fd-a333f7b4a1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:45.337601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a52c2f-7680-42ff-80fd-a333f7b4a1b5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:45.340313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24d89-6c18-4b35-be3a-766a718f9d1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:45.341932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24d89-6c18-4b35-be3a-766a718f9d1a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:45.351210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 938/1864] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:45.352755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9884f1a-4819-4e30-9cfe-6ed1cfc52822 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.353044 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9884f1a-4819-4e30-9cfe-6ed1cfc52822 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.353368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9884f1a-4819-4e30-9cfe-6ed1cfc52822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.380160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 927/1865] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:45 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:23:45.384192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a52c2f-7680-42ff-80fd-a333f7b4a1b5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.388962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e9884f1a-4819-4e30-9cfe-6ed1cfc52822 None None] MFA Rules not processed for user `3dc05026199e43368fe6d333215b0cec`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:45.390583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e9884f1a-4819-4e30-9cfe-6ed1cfc52822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:45.397986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a52c2f-7680-42ff-80fd-a333f7b4a1b5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.406195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac7580-2dff-4496-afc9-d77ed428c6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:45.407576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac7580-2dff-4496-afc9-d77ed428c6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.407989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac7580-2dff-4496-afc9-d77ed428c6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.408477 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac7580-2dff-4496-afc9-d77ed428c6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.409554 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac7580-2dff-4496-afc9-d77ed428c6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:45.412275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ac7580-2dff-4496-afc9-d77ed428c6bd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:45.435447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 928/1866] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:45.435863 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac7580-2dff-4496-afc9-d77ed428c6bd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.450717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac7580-2dff-4496-afc9-d77ed428c6bd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.459266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc367645-75e8-4c2a-8951-0d4b70d48fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:45.459998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc367645-75e8-4c2a-8951-0d4b70d48fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.461467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc367645-75e8-4c2a-8951-0d4b70d48fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.462026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc367645-75e8-4c2a-8951-0d4b70d48fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.463224 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc367645-75e8-4c2a-8951-0d4b70d48fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:45.466791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc367645-75e8-4c2a-8951-0d4b70d48fcc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:45.488504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 929/1867] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:45.494351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc367645-75e8-4c2a-8951-0d4b70d48fcc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.510158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc367645-75e8-4c2a-8951-0d4b70d48fcc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.522079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586532a-7990-4f03-aac3-26536c599154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:45.523487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586532a-7990-4f03-aac3-26536c599154 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.524185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586532a-7990-4f03-aac3-26536c599154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.524775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586532a-7990-4f03-aac3-26536c599154 admin admin] PATH_INFO: `/v3/projects/10a1d13bd76b425caead8798361e8fbd/users/d9fcc7a45cde48bb8e98997a0e403ca8/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.546434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 939/1868] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:45.562045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9884f1a-4819-4e30-9cfe-6ed1cfc52822 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.563578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586532a-7990-4f03-aac3-26536c599154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a1d13bd76b425caead8798361e8fbd, user_id=d9fcc7a45cde48bb8e98997a0e403ca8, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:45.567201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586532a-7990-4f03-aac3-26536c599154 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:45.575973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9884f1a-4819-4e30-9cfe-6ed1cfc52822 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.582012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a3db3-38be-434c-8f0c-0abd6ba35bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:45.582889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a3db3-38be-434c-8f0c-0abd6ba35bef service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.583142 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a3db3-38be-434c-8f0c-0abd6ba35bef service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.583438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a3db3-38be-434c-8f0c-0abd6ba35bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.586013 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 930/1869] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:45 2022] PUT /identity/v3/projects/10a1d13bd76b425caead8798361e8fbd/users/d9fcc7a45cde48bb8e98997a0e403ca8/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:45.597146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a3db3-38be-434c-8f0c-0abd6ba35bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:45.598844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a3db3-38be-434c-8f0c-0abd6ba35bef service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:45.599664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3ac09-5eb2-4143-99fe-78622612edf9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:45.600301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3ac09-5eb2-4143-99fe-78622612edf9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:45.601581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3ac09-5eb2-4143-99fe-78622612edf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:45.611940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 940/1870] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:45.658596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-48d3ac09-5eb2-4143-99fe-78622612edf9 None None] MFA Rules not processed for user `d9fcc7a45cde48bb8e98997a0e403ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:45.658596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-48d3ac09-5eb2-4143-99fe-78622612edf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:45.730785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:45.853214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 931/1871] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:45.855180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a3db3-38be-434c-8f0c-0abd6ba35bef service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:45.884690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a3db3-38be-434c-8f0c-0abd6ba35bef service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:45.907721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:46.050964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cff5ac-9fa6-49df-a1e2-a80c95755883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:46.052064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cff5ac-9fa6-49df-a1e2-a80c95755883 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.052672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cff5ac-9fa6-49df-a1e2-a80c95755883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.053209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cff5ac-9fa6-49df-a1e2-a80c95755883 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.054574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cff5ac-9fa6-49df-a1e2-a80c95755883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:46.059033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cff5ac-9fa6-49df-a1e2-a80c95755883 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:46.108076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 941/1872] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:45 2022] POST /identity/v3/projects => generated 379 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:23:46.112145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3ac09-5eb2-4143-99fe-78622612edf9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:46.125771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3ac09-5eb2-4143-99fe-78622612edf9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:46.223055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9e93-40a9-447c-b8eb-096d849ed671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:46.223671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9e93-40a9-447c-b8eb-096d849ed671 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.223947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9e93-40a9-447c-b8eb-096d849ed671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.224335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9e93-40a9-447c-b8eb-096d849ed671 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.225392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be9e93-40a9-447c-b8eb-096d849ed671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:46.230617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be9e93-40a9-447c-b8eb-096d849ed671 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:46.268822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 932/1873] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:46 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:23:46.270668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cff5ac-9fa6-49df-a1e2-a80c95755883 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:46.293350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cff5ac-9fa6-49df-a1e2-a80c95755883 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:46.300674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b5a42-06b7-4f9e-9a05-458d4a9aada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:46.300674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b5a42-06b7-4f9e-9a05-458d4a9aada7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.300674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b5a42-06b7-4f9e-9a05-458d4a9aada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.300674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b5a42-06b7-4f9e-9a05-458d4a9aada7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.301318 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b5a42-06b7-4f9e-9a05-458d4a9aada7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:46.303427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b5a42-06b7-4f9e-9a05-458d4a9aada7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:46.319805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 942/1874] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:23:46.321452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9e93-40a9-447c-b8eb-096d849ed671 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:46.339146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9e93-40a9-447c-b8eb-096d849ed671 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:46.348786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2919b-96a0-467b-b5ec-53bc0aa56c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:46.349967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2919b-96a0-467b-b5ec-53bc0aa56c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.350460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2919b-96a0-467b-b5ec-53bc0aa56c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.350956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2919b-96a0-467b-b5ec-53bc0aa56c53 admin admin] PATH_INFO: `/v3/projects/25c24930f23541e9baa3aad3fe2f299c/users/67cdd4a6acea4b6e8f72d4668fbfa667/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.372894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2919b-96a0-467b-b5ec-53bc0aa56c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c24930f23541e9baa3aad3fe2f299c, user_id=67cdd4a6acea4b6e8f72d4668fbfa667, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:46.375968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2919b-96a0-467b-b5ec-53bc0aa56c53 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:46.402629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 933/1875] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:46 2022] PUT /identity/v3/projects/25c24930f23541e9baa3aad3fe2f299c/users/67cdd4a6acea4b6e8f72d4668fbfa667/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:46.407497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b5a42-06b7-4f9e-9a05-458d4a9aada7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:46.420473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b5a42-06b7-4f9e-9a05-458d4a9aada7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:46.503949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31fd90-86dc-4b75-b948-d8ff169f1a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:46.504707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31fd90-86dc-4b75-b948-d8ff169f1a79 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.505060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31fd90-86dc-4b75-b948-d8ff169f1a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.505431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31fd90-86dc-4b75-b948-d8ff169f1a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.506875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31fd90-86dc-4b75-b948-d8ff169f1a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:46.509634 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31fd90-86dc-4b75-b948-d8ff169f1a79 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:46.534721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 943/1876] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:46.538840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2919b-96a0-467b-b5ec-53bc0aa56c53 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:46.551492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2919b-96a0-467b-b5ec-53bc0aa56c53 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:46.556578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6a003-79c2-4809-9256-d78d7ce56314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:46.558052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6a003-79c2-4809-9256-d78d7ce56314 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.558409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6a003-79c2-4809-9256-d78d7ce56314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.558743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6a003-79c2-4809-9256-d78d7ce56314 admin admin] PATH_INFO: `/v3/projects/25c24930f23541e9baa3aad3fe2f299c/users/67cdd4a6acea4b6e8f72d4668fbfa667/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.560840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6a003-79c2-4809-9256-d78d7ce56314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c24930f23541e9baa3aad3fe2f299c, user_id=67cdd4a6acea4b6e8f72d4668fbfa667, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:46.568478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6a003-79c2-4809-9256-d78d7ce56314 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:46.583249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 934/1877] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:46 2022] PUT /identity/v3/projects/25c24930f23541e9baa3aad3fe2f299c/users/67cdd4a6acea4b6e8f72d4668fbfa667/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:46.594129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f87d4-f96e-4f7f-bf7f-d3b63ee0c8d0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.594427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f87d4-f96e-4f7f-bf7f-d3b63ee0c8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.594710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f87d4-f96e-4f7f-bf7f-d3b63ee0c8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.625956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ce6f87d4-f96e-4f7f-bf7f-d3b63ee0c8d0 None None] MFA Rules not processed for user `67cdd4a6acea4b6e8f72d4668fbfa667`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:46.627201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ce6f87d4-f96e-4f7f-bf7f-d3b63ee0c8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:46.660189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6a003-79c2-4809-9256-d78d7ce56314 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:46.720539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 944/1878] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:46.736543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f87d4-f96e-4f7f-bf7f-d3b63ee0c8d0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:46.742082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:46.764638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6a003-79c2-4809-9256-d78d7ce56314 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:46.769762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:46.770460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.770740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.771016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] PATH_INFO: `/v3/users/9894395bab9c4e838db8185069be2def` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.772486 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9894395bab9c4e838db8185069be2def)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:46.774288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:46.820766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:46.835698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f87d4-f96e-4f7f-bf7f-d3b63ee0c8d0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:46.840453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ca673-7822-4fa7-82f2-721579a5d2c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:46.841073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ca673-7822-4fa7-82f2-721579a5d2c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.841409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ca673-7822-4fa7-82f2-721579a5d2c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.841859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ca673-7822-4fa7-82f2-721579a5d2c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.845577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9894395bab9c4e838db8185069be2def'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:46.852225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ca673-7822-4fa7-82f2-721579a5d2c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:46.853729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ca673-7822-4fa7-82f2-721579a5d2c8 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:46.860874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9894395bab9c4e838db8185069be2def'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:46.862163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9894395bab9c4e838db8185069be2def'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:46.865839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 945/1879] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:46.879760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9894395bab9c4e838db8185069be2def'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:46.891891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9894395bab9c4e838db8185069be2def'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:46.918462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 935/1880] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:46 2022] DELETE /identity/v3/users/9894395bab9c4e838db8185069be2def => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:46.923175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01952-4b9b-4a73-87e6-3e49dc243a5e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:46.923554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01952-4b9b-4a73-87e6-3e49dc243a5e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:46.923907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee01952-4b9b-4a73-87e6-3e49dc243a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:46.930000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:46.931362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:46.945613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fee01952-4b9b-4a73-87e6-3e49dc243a5e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:46.946986 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fee01952-4b9b-4a73-87e6-3e49dc243a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:47.023202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 946/1881] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:47.030316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c0583-8143-48a3-a213-46dfe666c64d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:47.034320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01952-4b9b-4a73-87e6-3e49dc243a5e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:47.034983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204dcaa-468a-441d-9aea-6a26eab3157f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:47.035591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204dcaa-468a-441d-9aea-6a26eab3157f service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:47.035873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204dcaa-468a-441d-9aea-6a26eab3157f service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:47.036143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204dcaa-468a-441d-9aea-6a26eab3157f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:47.036612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fee01952-4b9b-4a73-87e6-3e49dc243a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:47.039118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5204dcaa-468a-441d-9aea-6a26eab3157f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:47.118066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee01952-4b9b-4a73-87e6-3e49dc243a5e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:47.122872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdfd02-7bea-495a-a222-b1c69028ee39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:47.123763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdfd02-7bea-495a-a222-b1c69028ee39 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:47.124055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdfd02-7bea-495a-a222-b1c69028ee39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:47.124297 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdfd02-7bea-495a-a222-b1c69028ee39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:47.135009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fdfd02-7bea-495a-a222-b1c69028ee39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:47.138197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fdfd02-7bea-495a-a222-b1c69028ee39 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:47.149830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 947/1882] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:47.152727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204dcaa-468a-441d-9aea-6a26eab3157f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:47.154887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204dcaa-468a-441d-9aea-6a26eab3157f service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:47.172471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 936/1883] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:47.603412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdfd02-7bea-495a-a222-b1c69028ee39 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:47.605511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-93fdfd02-7bea-495a-a222-b1c69028ee39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:47.646012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdfd02-7bea-495a-a222-b1c69028ee39 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:47.672727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:47.672727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:47.672727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:47.672727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] PATH_INFO: `/v3/projects/82e150d42bb44293b184c3d5c5fd2dee` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:47.673698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e150d42bb44293b184c3d5c5fd2dee)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:47.677804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:47.771272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:47.779581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e150d42bb44293b184c3d5c5fd2dee', 'request_id': 'req-80f93799-d36d-41f8-a00c-f0393f6b9024'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:47.792895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e150d42bb44293b184c3d5c5fd2dee', 'request_id': 'req-80f93799-d36d-41f8-a00c-f0393f6b9024'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:47.809001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e150d42bb44293b184c3d5c5fd2dee', 'request_id': 'req-80f93799-d36d-41f8-a00c-f0393f6b9024'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:47.837437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] The token cache is being invalidate because project 82e150d42bb44293b184c3d5c5fd2dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:47.837759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:47.839256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:47.864487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 948/1884] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:47 2022] DELETE /identity/v3/projects/82e150d42bb44293b184c3d5c5fd2dee => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:47.868555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204dcaa-468a-441d-9aea-6a26eab3157f service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:47.886918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204dcaa-468a-441d-9aea-6a26eab3157f service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:48.020073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:48.029700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:48.030570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:48.030807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:48.031432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] PATH_INFO: `/v3/users/3dc05026199e43368fe6d333215b0cec` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:48.034288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc05026199e43368fe6d333215b0cec)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:48.036830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:48.163450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc05026199e43368fe6d333215b0cec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.187402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc05026199e43368fe6d333215b0cec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.198005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc05026199e43368fe6d333215b0cec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.205947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc05026199e43368fe6d333215b0cec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.316283 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e72a1987-269d-4228-a959-1465df980ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc05026199e43368fe6d333215b0cec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.329449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 937/1885] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:47 2022] DELETE /identity/v3/users/3dc05026199e43368fe6d333215b0cec => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:48.676352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:48.679314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:48.700181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f93799-d36d-41f8-a00c-f0393f6b9024 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:48.780325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:48.781012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:48.781298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:48.781571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] PATH_INFO: `/v3/projects/780ed9721711450b98b8b32f3018cb53` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:48.782851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780ed9721711450b98b8b32f3018cb53)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:48.784818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:48.844516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780ed9721711450b98b8b32f3018cb53', 'request_id': 'req-c2afa878-c426-4d0b-8c23-44ebafd5b317'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.855897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780ed9721711450b98b8b32f3018cb53', 'request_id': 'req-c2afa878-c426-4d0b-8c23-44ebafd5b317'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.865822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780ed9721711450b98b8b32f3018cb53', 'request_id': 'req-c2afa878-c426-4d0b-8c23-44ebafd5b317'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.880258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] The token cache is being invalidate because project 780ed9721711450b98b8b32f3018cb53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:48.880662 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.881242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2afa878-c426-4d0b-8c23-44ebafd5b317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:48.883343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 949/1886] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:48 2022] DELETE /identity/v3/projects/780ed9721711450b98b8b32f3018cb53 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:48.894595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b780a-f18c-4376-8bdd-da429bce8a60 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:48.894595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b780a-f18c-4376-8bdd-da429bce8a60 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:48.894595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b780a-f18c-4376-8bdd-da429bce8a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:48.918397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f94b780a-f18c-4376-8bdd-da429bce8a60 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:48.919645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f94b780a-f18c-4376-8bdd-da429bce8a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:49.020942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 938/1887] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:49.023354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad3fca-81d6-48a4-83be-a59b2249d790 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:49.023578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad3fca-81d6-48a4-83be-a59b2249d790 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:49.024271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad3fca-81d6-48a4-83be-a59b2249d790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:49.049324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-63ad3fca-81d6-48a4-83be-a59b2249d790 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:49.051466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-63ad3fca-81d6-48a4-83be-a59b2249d790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:49.093338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 950/1888] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:49.093739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b780a-f18c-4376-8bdd-da429bce8a60 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:49.108620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b780a-f18c-4376-8bdd-da429bce8a60 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:49.119418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438182ef-a1a1-49c5-9954-fbf55fab9b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:49.121665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438182ef-a1a1-49c5-9954-fbf55fab9b2a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:49.121949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438182ef-a1a1-49c5-9954-fbf55fab9b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:49.122388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438182ef-a1a1-49c5-9954-fbf55fab9b2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:49.122887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438182ef-a1a1-49c5-9954-fbf55fab9b2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:49.123307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438182ef-a1a1-49c5-9954-fbf55fab9b2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:49.125756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438182ef-a1a1-49c5-9954-fbf55fab9b2a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:49.146658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 939/1889] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:49.152515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad3fca-81d6-48a4-83be-a59b2249d790 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:49.163376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad3fca-81d6-48a4-83be-a59b2249d790 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:49.175429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5e2f3-4c7c-49d2-aecb-71a07bcdd928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:49.176110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5e2f3-4c7c-49d2-aecb-71a07bcdd928 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:49.176428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5e2f3-4c7c-49d2-aecb-71a07bcdd928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:49.176701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5e2f3-4c7c-49d2-aecb-71a07bcdd928 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:49.177353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5e2f3-4c7c-49d2-aecb-71a07bcdd928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:49.179321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5e2f3-4c7c-49d2-aecb-71a07bcdd928 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:49.219150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 951/1890] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:49 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:49.223468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438182ef-a1a1-49c5-9954-fbf55fab9b2a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:49.248273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438182ef-a1a1-49c5-9954-fbf55fab9b2a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:49.372602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5391e-54be-490e-a9fc-b57707eec54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:49.373526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5391e-54be-490e-a9fc-b57707eec54f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:49.373890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5391e-54be-490e-a9fc-b57707eec54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:49.374170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5391e-54be-490e-a9fc-b57707eec54f admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:49.374818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5391e-54be-490e-a9fc-b57707eec54f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:49.387468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5391e-54be-490e-a9fc-b57707eec54f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:49.409958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:49.442370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 940/1891] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:49 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:49.460782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5e2f3-4c7c-49d2-aecb-71a07bcdd928 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:49.476616 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5e2f3-4c7c-49d2-aecb-71a07bcdd928 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:49.504195 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7f39c-e675-4071-b66c-0c982c731428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:49.506082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7f39c-e675-4071-b66c-0c982c731428 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:49.507573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7f39c-e675-4071-b66c-0c982c731428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:49.507868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7f39c-e675-4071-b66c-0c982c731428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:49.511205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7f39c-e675-4071-b66c-0c982c731428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:49.514087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7f39c-e675-4071-b66c-0c982c731428 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:49.534032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 952/1892] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:49.537923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5391e-54be-490e-a9fc-b57707eec54f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:49.551829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5391e-54be-490e-a9fc-b57707eec54f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:49.556749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710a9a0-75de-4441-883d-59157eb7dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:49.558040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710a9a0-75de-4441-883d-59157eb7dfa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:49.558432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710a9a0-75de-4441-883d-59157eb7dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:49.558945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710a9a0-75de-4441-883d-59157eb7dfa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:49.559766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710a9a0-75de-4441-883d-59157eb7dfa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:49.561712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710a9a0-75de-4441-883d-59157eb7dfa7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:49.577641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 941/1893] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:49.579267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7f39c-e675-4071-b66c-0c982c731428 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:49.594302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7f39c-e675-4071-b66c-0c982c731428 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:49.605239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa46ac-6189-4e16-9394-a25d2470314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:49.605239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa46ac-6189-4e16-9394-a25d2470314a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:49.605239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa46ac-6189-4e16-9394-a25d2470314a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:49.605239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa46ac-6189-4e16-9394-a25d2470314a admin admin] PATH_INFO: `/v3/projects/b6d71194223d4ede8cb4e8e69cc0948d/users/24ff15f62e304fbda0bb95854be7690a/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:49.636319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa46ac-6189-4e16-9394-a25d2470314a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d71194223d4ede8cb4e8e69cc0948d, user_id=24ff15f62e304fbda0bb95854be7690a, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:49.639333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa46ac-6189-4e16-9394-a25d2470314a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:49.652208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:49.658451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 953/1894] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:49 2022] PUT /identity/v3/projects/b6d71194223d4ede8cb4e8e69cc0948d/users/24ff15f62e304fbda0bb95854be7690a/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:49.669519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d70d65-3edc-47f3-8aa2-0b169a5620dc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:49.669815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d70d65-3edc-47f3-8aa2-0b169a5620dc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:49.670115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d70d65-3edc-47f3-8aa2-0b169a5620dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:49.712434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-39d70d65-3edc-47f3-8aa2-0b169a5620dc None None] MFA Rules not processed for user `24ff15f62e304fbda0bb95854be7690a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:49.714341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-39d70d65-3edc-47f3-8aa2-0b169a5620dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:49.805274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 942/1895] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:49.819057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa46ac-6189-4e16-9394-a25d2470314a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:49.820812 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cafa46ac-6189-4e16-9394-a25d2470314a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:49.919775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa46ac-6189-4e16-9394-a25d2470314a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:49.929578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897d4ff-1b08-4855-b42d-bd1834a0f628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:49.930216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897d4ff-1b08-4855-b42d-bd1834a0f628 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:49.930501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897d4ff-1b08-4855-b42d-bd1834a0f628 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:49.930786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897d4ff-1b08-4855-b42d-bd1834a0f628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:49.942554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897d4ff-1b08-4855-b42d-bd1834a0f628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:49.947608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897d4ff-1b08-4855-b42d-bd1834a0f628 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:49.961951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 954/1896] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:49.992777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d70d65-3edc-47f3-8aa2-0b169a5620dc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:50.112418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d70d65-3edc-47f3-8aa2-0b169a5620dc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:50.118712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:50.120927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:50.121532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:50.122010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] PATH_INFO: `/v3/users/24ff15f62e304fbda0bb95854be7690a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:50.123517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ff15f62e304fbda0bb95854be7690a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:50.126011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:50.204064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ff15f62e304fbda0bb95854be7690a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:50.215389 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ff15f62e304fbda0bb95854be7690a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:50.216137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ff15f62e304fbda0bb95854be7690a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:50.238187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ff15f62e304fbda0bb95854be7690a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:50.242254 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:50.251014 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ff15f62e304fbda0bb95854be7690a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:50.265118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 943/1897] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:49 2022] DELETE /identity/v3/users/24ff15f62e304fbda0bb95854be7690a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:50.272266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0316fe-55b7-41d8-9037-c1c3ed1ffeb8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:50.273217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0316fe-55b7-41d8-9037-c1c3ed1ffeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:50.273502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0316fe-55b7-41d8-9037-c1c3ed1ffeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:50.302290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2e0316fe-55b7-41d8-9037-c1c3ed1ffeb8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:50.303516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e0316fe-55b7-41d8-9037-c1c3ed1ffeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:50.373676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 955/1898] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:50.383788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:50.386132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:50.453209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e87a5c-6838-4791-bdff-94b59a0ae714 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:50.456966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc8d7b-36a2-4030-9c31-b6a39f64f374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:50.457543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc8d7b-36a2-4030-9c31-b6a39f64f374 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:50.457826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc8d7b-36a2-4030-9c31-b6a39f64f374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:50.458096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc8d7b-36a2-4030-9c31-b6a39f64f374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:50.471869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc8d7b-36a2-4030-9c31-b6a39f64f374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:50.473556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc8d7b-36a2-4030-9c31-b6a39f64f374 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:50.484871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 944/1899] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:50.797080 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0316fe-55b7-41d8-9037-c1c3ed1ffeb8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:50.808962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e0316fe-55b7-41d8-9037-c1c3ed1ffeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:50.939076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0316fe-55b7-41d8-9037-c1c3ed1ffeb8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:50.943405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc8d7b-36a2-4030-9c31-b6a39f64f374 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:50.944919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-67fc8d7b-36a2-4030-9c31-b6a39f64f374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:50.949108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4c813-e079-406d-b66d-294a854cea4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:50.950144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4c813-e079-406d-b66d-294a854cea4c service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:50.950576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4c813-e079-406d-b66d-294a854cea4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:50.951000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4c813-e079-406d-b66d-294a854cea4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:50.953775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f2a4c813-e079-406d-b66d-294a854cea4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:50.970564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc8d7b-36a2-4030-9c31-b6a39f64f374 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:50.996545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:50.997355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:50.997742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:50.998126 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] PATH_INFO: `/v3/users/79467cf25fa34db08a8ba53afedc75fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:51.003507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79467cf25fa34db08a8ba53afedc75fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:51.005570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:51.097080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79467cf25fa34db08a8ba53afedc75fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.112989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79467cf25fa34db08a8ba53afedc75fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.114056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79467cf25fa34db08a8ba53afedc75fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.127594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4c813-e079-406d-b66d-294a854cea4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:51.131835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4c813-e079-406d-b66d-294a854cea4c service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:51.136973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:51.139616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79467cf25fa34db08a8ba53afedc75fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.154905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79467cf25fa34db08a8ba53afedc75fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.157408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 956/1900] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:51.218333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 945/1901] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:50 2022] DELETE /identity/v3/users/79467cf25fa34db08a8ba53afedc75fd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:51.223147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575104ee-a3d7-4efc-9bbc-30cf55667c91 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:51.224098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575104ee-a3d7-4efc-9bbc-30cf55667c91 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:51.224990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575104ee-a3d7-4efc-9bbc-30cf55667c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:51.300966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-575104ee-a3d7-4efc-9bbc-30cf55667c91 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:51.302289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-575104ee-a3d7-4efc-9bbc-30cf55667c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:51.422434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:51.425651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:51.554380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 957/1902] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:51.567860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575104ee-a3d7-4efc-9bbc-30cf55667c91 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:51.571115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-575104ee-a3d7-4efc-9bbc-30cf55667c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:51.656994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82c159-322f-4667-8f75-4bb82cc94c05 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:51.673673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:51.674949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:51.675321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:51.676130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] PATH_INFO: `/v3/projects/b6d71194223d4ede8cb4e8e69cc0948d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:51.681047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d71194223d4ede8cb4e8e69cc0948d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:51.690664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:51.711482 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:51.739897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575104ee-a3d7-4efc-9bbc-30cf55667c91 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:51.749742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ccea3-22c6-4166-a342-77ee6874e437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:51.751925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ccea3-22c6-4166-a342-77ee6874e437 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:51.752320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ccea3-22c6-4166-a342-77ee6874e437 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:51.755012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ccea3-22c6-4166-a342-77ee6874e437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:51.781687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ccea3-22c6-4166-a342-77ee6874e437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:51.783195 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ccea3-22c6-4166-a342-77ee6874e437 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:51.814571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d71194223d4ede8cb4e8e69cc0948d', 'request_id': 'req-3d1d2d7d-2756-497d-a64b-1f249d3172f0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.822372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 958/1903] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:51.841739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d71194223d4ede8cb4e8e69cc0948d', 'request_id': 'req-3d1d2d7d-2756-497d-a64b-1f249d3172f0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.875417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d71194223d4ede8cb4e8e69cc0948d', 'request_id': 'req-3d1d2d7d-2756-497d-a64b-1f249d3172f0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.906259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] The token cache is being invalidate because project b6d71194223d4ede8cb4e8e69cc0948d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:51.906584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.907295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d1d2d7d-2756-497d-a64b-1f249d3172f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:51.908817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 946/1904] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:51 2022] DELETE /identity/v3/projects/b6d71194223d4ede8cb4e8e69cc0948d => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:51.914060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303ff26-3bc8-400f-b138-b11e7665a680 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:51.914555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303ff26-3bc8-400f-b138-b11e7665a680 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:51.915814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303ff26-3bc8-400f-b138-b11e7665a680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:51.974064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4303ff26-3bc8-400f-b138-b11e7665a680 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:51.975297 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4303ff26-3bc8-400f-b138-b11e7665a680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:52.133331 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 959/1905] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:52.143112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a849a8b-82fb-4449-b06a-704af39865cf None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:52.143983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a849a8b-82fb-4449-b06a-704af39865cf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:52.143983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a849a8b-82fb-4449-b06a-704af39865cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:52.205463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1a849a8b-82fb-4449-b06a-704af39865cf None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:52.209951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1a849a8b-82fb-4449-b06a-704af39865cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:52.237202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 947/1906] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:52.238933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303ff26-3bc8-400f-b138-b11e7665a680 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:52.252954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303ff26-3bc8-400f-b138-b11e7665a680 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:52.268064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74d170-7ec3-4ddd-9984-31bc7ea59777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:52.268734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74d170-7ec3-4ddd-9984-31bc7ea59777 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:52.269006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74d170-7ec3-4ddd-9984-31bc7ea59777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:52.269278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74d170-7ec3-4ddd-9984-31bc7ea59777 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:52.269944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74d170-7ec3-4ddd-9984-31bc7ea59777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:52.270243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74d170-7ec3-4ddd-9984-31bc7ea59777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:52.271891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74d170-7ec3-4ddd-9984-31bc7ea59777 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:52.296250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 960/1907] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:52.300917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a849a8b-82fb-4449-b06a-704af39865cf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:52.317609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a849a8b-82fb-4449-b06a-704af39865cf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:52.335303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b36b6a-910c-4ad7-98e8-de91caed20b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:52.335637 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b36b6a-910c-4ad7-98e8-de91caed20b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:52.335923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b36b6a-910c-4ad7-98e8-de91caed20b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:52.336197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b36b6a-910c-4ad7-98e8-de91caed20b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:52.336894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b36b6a-910c-4ad7-98e8-de91caed20b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:52.340038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b36b6a-910c-4ad7-98e8-de91caed20b1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:52.426120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 948/1908] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:52 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:52.430327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74d170-7ec3-4ddd-9984-31bc7ea59777 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:52.449641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74d170-7ec3-4ddd-9984-31bc7ea59777 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:52.514482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:52.593104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee225a6-dd1f-4345-aa8a-57bd0c4dc30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:52.594247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee225a6-dd1f-4345-aa8a-57bd0c4dc30d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:52.596193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee225a6-dd1f-4345-aa8a-57bd0c4dc30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:52.596768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee225a6-dd1f-4345-aa8a-57bd0c4dc30d admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:52.597684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee225a6-dd1f-4345-aa8a-57bd0c4dc30d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:52.600049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee225a6-dd1f-4345-aa8a-57bd0c4dc30d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:52.702256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 961/1909] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:52 2022] POST /identity/v3/users => generated 292 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:52.706332 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b36b6a-910c-4ad7-98e8-de91caed20b1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:52.727582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b36b6a-910c-4ad7-98e8-de91caed20b1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:52.737134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03d54-1312-405a-a87d-2e17a05f118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:52.738625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03d54-1312-405a-a87d-2e17a05f118b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:52.739409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03d54-1312-405a-a87d-2e17a05f118b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:52.740264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03d54-1312-405a-a87d-2e17a05f118b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:52.742002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a03d54-1312-405a-a87d-2e17a05f118b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:52.744305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a03d54-1312-405a-a87d-2e17a05f118b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:52.764333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 949/1910] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:52.774451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee225a6-dd1f-4345-aa8a-57bd0c4dc30d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:52.798636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee225a6-dd1f-4345-aa8a-57bd0c4dc30d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:52.813599 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4ba61-2969-40f5-b4dc-3d6a1794133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:52.815705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4ba61-2969-40f5-b4dc-3d6a1794133e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:52.816736 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4ba61-2969-40f5-b4dc-3d6a1794133e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:52.817020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4ba61-2969-40f5-b4dc-3d6a1794133e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:52.817712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4ba61-2969-40f5-b4dc-3d6a1794133e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:52.824168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4ba61-2969-40f5-b4dc-3d6a1794133e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:52.850945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 962/1911] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:52.857080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03d54-1312-405a-a87d-2e17a05f118b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:52.885268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a03d54-1312-405a-a87d-2e17a05f118b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:52.902216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564928e-a395-42d5-894f-c9941ee32a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:52.903418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564928e-a395-42d5-894f-c9941ee32a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:52.922281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564928e-a395-42d5-894f-c9941ee32a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:52.922482 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564928e-a395-42d5-894f-c9941ee32a94 admin admin] PATH_INFO: `/v3/projects/0c537f4aaa5c43119ab973f688215a6e/users/c180ccad2a7f4ab1b50a16a4514749ca/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:52.963367 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564928e-a395-42d5-894f-c9941ee32a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c537f4aaa5c43119ab973f688215a6e, user_id=c180ccad2a7f4ab1b50a16a4514749ca, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:52.975059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564928e-a395-42d5-894f-c9941ee32a94 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:53.000585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 950/1912] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:52 2022] PUT /identity/v3/projects/0c537f4aaa5c43119ab973f688215a6e/users/c180ccad2a7f4ab1b50a16a4514749ca/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:53.004372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f60cdf-cb40-4dad-9def-a38476d8b101 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:53.005372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f60cdf-cb40-4dad-9def-a38476d8b101 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:53.005945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f60cdf-cb40-4dad-9def-a38476d8b101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:53.045478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e0f60cdf-cb40-4dad-9def-a38476d8b101 None None] MFA Rules not processed for user `c180ccad2a7f4ab1b50a16a4514749ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:53.047076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e0f60cdf-cb40-4dad-9def-a38476d8b101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:53.128868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564928e-a395-42d5-894f-c9941ee32a94 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:53.131601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6564928e-a395-42d5-894f-c9941ee32a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:53.164731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 963/1913] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:53.176061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f60cdf-cb40-4dad-9def-a38476d8b101 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:53.177587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e0f60cdf-cb40-4dad-9def-a38476d8b101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:53.247447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564928e-a395-42d5-894f-c9941ee32a94 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:53.254446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:53.255117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:53.255398 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:53.255675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] PATH_INFO: `/v3/projects/1a8aace3fc7348d2877c1b96e1df9bf4` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:53.256855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a8aace3fc7348d2877c1b96e1df9bf4)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:53.258805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:53.268585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f60cdf-cb40-4dad-9def-a38476d8b101 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:53.281354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07b757-5fdd-4680-94e5-745121bbf57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:53.282008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07b757-5fdd-4680-94e5-745121bbf57a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:53.282369 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07b757-5fdd-4680-94e5-745121bbf57a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:53.282669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07b757-5fdd-4680-94e5-745121bbf57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:53.294042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07b757-5fdd-4680-94e5-745121bbf57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:53.295631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07b757-5fdd-4680-94e5-745121bbf57a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:53.305320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:53.306604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 964/1914] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:53.316064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a8aace3fc7348d2877c1b96e1df9bf4', 'request_id': 'req-6bb38637-879a-4cd0-8974-220ded1baefc'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.322230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a8aace3fc7348d2877c1b96e1df9bf4', 'request_id': 'req-6bb38637-879a-4cd0-8974-220ded1baefc'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.345397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a8aace3fc7348d2877c1b96e1df9bf4', 'request_id': 'req-6bb38637-879a-4cd0-8974-220ded1baefc'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.361827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] The token cache is being invalidate because project 1a8aace3fc7348d2877c1b96e1df9bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:53.362126 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.363185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.368455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 951/1915] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:53 2022] DELETE /identity/v3/projects/1a8aace3fc7348d2877c1b96e1df9bf4 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:53.372100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07b757-5fdd-4680-94e5-745121bbf57a service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:53.388905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07b757-5fdd-4680-94e5-745121bbf57a service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:53.454772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:53.512457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:53.514168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:53.515325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:53.515653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] PATH_INFO: `/v3/users/d9fcc7a45cde48bb8e98997a0e403ca8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:53.517380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9fcc7a45cde48bb8e98997a0e403ca8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:53.520998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:53.629886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9fcc7a45cde48bb8e98997a0e403ca8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.631255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9fcc7a45cde48bb8e98997a0e403ca8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.632763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9fcc7a45cde48bb8e98997a0e403ca8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.659881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9fcc7a45cde48bb8e98997a0e403ca8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.681156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9fcc7a45cde48bb8e98997a0e403ca8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:53.707180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 965/1916] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:53 2022] DELETE /identity/v3/users/d9fcc7a45cde48bb8e98997a0e403ca8 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:54.359033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:54.365635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:54.392152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb38637-879a-4cd0-8974-220ded1baefc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:54.488542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:54.490023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:54.490486 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:54.491278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] PATH_INFO: `/v3/projects/10a1d13bd76b425caead8798361e8fbd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:54.493525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a1d13bd76b425caead8798361e8fbd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:54.499030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:54.565592 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:54.567674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:54.580571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a1d13bd76b425caead8798361e8fbd', 'request_id': 'req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.590201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a1d13bd76b425caead8798361e8fbd', 'request_id': 'req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.594560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926343b-84c4-41f5-919d-398989a6148a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:54.600545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a1d13bd76b425caead8798361e8fbd', 'request_id': 'req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.618963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] The token cache is being invalidate because project 10a1d13bd76b425caead8798361e8fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:54.620012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.621443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.626504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 952/1917] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:54 2022] DELETE /identity/v3/projects/10a1d13bd76b425caead8798361e8fbd => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:54.627723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:54.629834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:54.718852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:54.728335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:54.731342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:54.731819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:54.732539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] PATH_INFO: `/v3/users/db64cfd432db40ce86d5e3cfd39f778d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:54.734228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db64cfd432db40ce86d5e3cfd39f778d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:54.737216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:54.747523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde047c9-d8a6-4a1a-8e97-35d44b922ef7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:54.753055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:54.753763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:54.754049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:54.754327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] PATH_INFO: `/v3/users/67cdd4a6acea4b6e8f72d4668fbfa667` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:54.755673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67cdd4a6acea4b6e8f72d4668fbfa667)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:54.757843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:54.830793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db64cfd432db40ce86d5e3cfd39f778d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.831622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db64cfd432db40ce86d5e3cfd39f778d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.833324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db64cfd432db40ce86d5e3cfd39f778d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.843005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67cdd4a6acea4b6e8f72d4668fbfa667'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.848956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db64cfd432db40ce86d5e3cfd39f778d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.853026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67cdd4a6acea4b6e8f72d4668fbfa667'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.854450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67cdd4a6acea4b6e8f72d4668fbfa667'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.858357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:54.871648 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67cdd4a6acea4b6e8f72d4668fbfa667'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.875145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db64cfd432db40ce86d5e3cfd39f778d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.887836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da828f67-8b58-4e57-938a-206c25316241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67cdd4a6acea4b6e8f72d4668fbfa667'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:54.900606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 953/1918] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:54 2022] DELETE /identity/v3/users/67cdd4a6acea4b6e8f72d4668fbfa667 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:54.908182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 966/1919] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:54 2022] DELETE /identity/v3/users/db64cfd432db40ce86d5e3cfd39f778d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:54.910361 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49759591-8023-4302-b9dd-e8b53e192a91 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:54.912505 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49759591-8023-4302-b9dd-e8b53e192a91 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:54.912799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49759591-8023-4302-b9dd-e8b53e192a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:54.945874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-49759591-8023-4302-b9dd-e8b53e192a91 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:54.947124 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-49759591-8023-4302-b9dd-e8b53e192a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:55.071636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 954/1920] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:55.084361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:55.086912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:55.212987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126dc1d5-602f-4499-8d22-5b853e31fb81 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:55.219544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652b0e-f81c-4d0d-9069-f8b15233087c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:55.222304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652b0e-f81c-4d0d-9069-f8b15233087c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:55.223054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652b0e-f81c-4d0d-9069-f8b15233087c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:55.223669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652b0e-f81c-4d0d-9069-f8b15233087c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:55.241025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e652b0e-f81c-4d0d-9069-f8b15233087c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:55.244311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e652b0e-f81c-4d0d-9069-f8b15233087c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:55.250721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594913-0cc4-467e-ae76-8c52a46bb2bb None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:55.251191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594913-0cc4-467e-ae76-8c52a46bb2bb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:55.251504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e594913-0cc4-467e-ae76-8c52a46bb2bb None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:55.253310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 955/1921] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:23:55.274817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e684-5ccb-48ad-81fb-3d92089c0f6a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:55.274817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e684-5ccb-48ad-81fb-3d92089c0f6a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:55.274817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e684-5ccb-48ad-81fb-3d92089c0f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:55.276599 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 967/1922] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:55.308354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5a22e684-5ccb-48ad-81fb-3d92089c0f6a None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:55.310952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5a22e684-5ccb-48ad-81fb-3d92089c0f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:55.523710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 956/1923] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:23:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:55.715172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652b0e-f81c-4d0d-9069-f8b15233087c service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:55.718321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e652b0e-f81c-4d0d-9069-f8b15233087c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:55.739804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652b0e-f81c-4d0d-9069-f8b15233087c service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:55.748107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508af3d-2950-4bb6-b69c-67185f7bc76a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:55.748791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508af3d-2950-4bb6-b69c-67185f7bc76a service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:55.749071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508af3d-2950-4bb6-b69c-67185f7bc76a service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:55.749340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508af3d-2950-4bb6-b69c-67185f7bc76a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:55.763344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22e684-5ccb-48ad-81fb-3d92089c0f6a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:55.764422 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508af3d-2950-4bb6-b69c-67185f7bc76a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:55.764977 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5a22e684-5ccb-48ad-81fb-3d92089c0f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:55.766051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508af3d-2950-4bb6-b69c-67185f7bc76a service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:55.776537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:55.780037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 968/1924] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:55.785302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22e684-5ccb-48ad-81fb-3d92089c0f6a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:55.793914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:55.794965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:55.795492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:55.796197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] PATH_INFO: `/v3/projects/25c24930f23541e9baa3aad3fe2f299c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:55.797826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c24930f23541e9baa3aad3fe2f299c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:55.799942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:55.912025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c24930f23541e9baa3aad3fe2f299c', 'request_id': 'req-f6c1228d-c5ce-4240-914e-e8df7238d55a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:55.929959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c24930f23541e9baa3aad3fe2f299c', 'request_id': 'req-f6c1228d-c5ce-4240-914e-e8df7238d55a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:55.943699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c24930f23541e9baa3aad3fe2f299c', 'request_id': 'req-f6c1228d-c5ce-4240-914e-e8df7238d55a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:55.945161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508af3d-2950-4bb6-b69c-67185f7bc76a service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:55.946997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9508af3d-2950-4bb6-b69c-67185f7bc76a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:55.976517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508af3d-2950-4bb6-b69c-67185f7bc76a service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:55.983684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] The token cache is being invalidate because project 25c24930f23541e9baa3aad3fe2f299c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:55.983979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:55.985413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6c1228d-c5ce-4240-914e-e8df7238d55a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:55.997893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 957/1925] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:55 2022] DELETE /identity/v3/projects/25c24930f23541e9baa3aad3fe2f299c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:56.001126 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbea054-bd3a-4a35-8ef5-011f8325495a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:56.001431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbea054-bd3a-4a35-8ef5-011f8325495a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:56.003552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbea054-bd3a-4a35-8ef5-011f8325495a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:56.057519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9bbea054-bd3a-4a35-8ef5-011f8325495a None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:56.062491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9bbea054-bd3a-4a35-8ef5-011f8325495a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:56.134790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:56.136633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:56.137226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:56.137568 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] PATH_INFO: `/v3/projects/f75d13b095e34c368cda8792ff7a553c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:56.139235 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f75d13b095e34c368cda8792ff7a553c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:56.142384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:56.155643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 958/1926] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:56.165048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bd667-dff6-46f7-8b46-8325af2e7202 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:56.165467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bd667-dff6-46f7-8b46-8325af2e7202 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:56.166138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bd667-dff6-46f7-8b46-8325af2e7202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:56.202160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7b1bd667-dff6-46f7-8b46-8325af2e7202 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:56.204039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7b1bd667-dff6-46f7-8b46-8325af2e7202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:56.216861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 959/1927] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:56.222822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bd667-dff6-46f7-8b46-8325af2e7202 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:56.236506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bd667-dff6-46f7-8b46-8325af2e7202 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:56.248561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f75d13b095e34c368cda8792ff7a553c', 'request_id': 'req-de97690a-8758-43e1-9b43-879e143ac0c8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.262869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f75d13b095e34c368cda8792ff7a553c', 'request_id': 'req-de97690a-8758-43e1-9b43-879e143ac0c8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.287566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f75d13b095e34c368cda8792ff7a553c', 'request_id': 'req-de97690a-8758-43e1-9b43-879e143ac0c8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.308532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:56.321625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] The token cache is being invalidate because project f75d13b095e34c368cda8792ff7a553c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:56.324206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.326270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.334736 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 969/1928] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:55 2022] DELETE /identity/v3/projects/f75d13b095e34c368cda8792ff7a553c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:56.336683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:56.342434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:56.452661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:56.454458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:56.454745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:56.456511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:56.458595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:56.459333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:56.465333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:56.500770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 960/1929] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:56.501808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:56.508258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:56.537926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5da6d4-cb92-45fb-bf0a-94b351bc7dc0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:56.550744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:56.551872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:56.554559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:56.555396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] PATH_INFO: `/v3/users/1158e4b2c6ff4009bbc5e85edba7d986` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:56.560867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1158e4b2c6ff4009bbc5e85edba7d986)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:56.563172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97690a-8758-43e1-9b43-879e143ac0c8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:56.573849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:56.576244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:56.577739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:56.579086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] PATH_INFO: `/v3/users/c180ccad2a7f4ab1b50a16a4514749ca` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:56.580269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:56.581840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c180ccad2a7f4ab1b50a16a4514749ca)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:56.586285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:56.631927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:56.708564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1158e4b2c6ff4009bbc5e85edba7d986'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.722182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1158e4b2c6ff4009bbc5e85edba7d986'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.723614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1158e4b2c6ff4009bbc5e85edba7d986'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.762258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1158e4b2c6ff4009bbc5e85edba7d986'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.767683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c180ccad2a7f4ab1b50a16a4514749ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.768495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c180ccad2a7f4ab1b50a16a4514749ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.769145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c180ccad2a7f4ab1b50a16a4514749ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.802590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c180ccad2a7f4ab1b50a16a4514749ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.808212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1158e4b2c6ff4009bbc5e85edba7d986'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.821467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 961/1930] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:56 2022] DELETE /identity/v3/users/1158e4b2c6ff4009bbc5e85edba7d986 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:56.823649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:56.828088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:56.847413 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11fd767f-895b-4448-9f7b-645d62fecfaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c180ccad2a7f4ab1b50a16a4514749ca'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:56.893716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 970/1931] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:56 2022] DELETE /identity/v3/users/c180ccad2a7f4ab1b50a16a4514749ca => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:56.900270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb8254-f434-4343-b690-3b29a0e58835 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:56.902608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb8254-f434-4343-b690-3b29a0e58835 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:56.903048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb8254-f434-4343-b690-3b29a0e58835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:56.949490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-52fb8254-f434-4343-b690-3b29a0e58835 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:56.950849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-52fb8254-f434-4343-b690-3b29a0e58835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:56.960211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075cec1-9f50-4e39-b6a6-ce20781063ab admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.069658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e907759-8ae7-444f-8453-0168f7ffe996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.070347 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907759-8ae7-444f-8453-0168f7ffe996 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.071174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907759-8ae7-444f-8453-0168f7ffe996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.071594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907759-8ae7-444f-8453-0168f7ffe996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.072437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e907759-8ae7-444f-8453-0168f7ffe996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.074927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e907759-8ae7-444f-8453-0168f7ffe996 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.128468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 971/1932] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:57.143233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb8254-f434-4343-b690-3b29a0e58835 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:57.147009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-52fb8254-f434-4343-b690-3b29a0e58835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:57.151007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 962/1933] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:56 2022] POST /identity/v3/projects => generated 375 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:57.156417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e907759-8ae7-444f-8453-0168f7ffe996 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:57.192757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:57.197196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:57.256337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb8254-f434-4343-b690-3b29a0e58835 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.258802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e907759-8ae7-444f-8453-0168f7ffe996 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.264703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa8ec1-97a1-48b3-973f-1ec797ef2ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.265502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa8ec1-97a1-48b3-973f-1ec797ef2ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.265719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa8ec1-97a1-48b3-973f-1ec797ef2ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.266047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa8ec1-97a1-48b3-973f-1ec797ef2ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.267242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa8ec1-97a1-48b3-973f-1ec797ef2ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.268931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa8ec1-97a1-48b3-973f-1ec797ef2ea0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.303286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 963/1934] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:57 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:57.308682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa8ec1-97a1-48b3-973f-1ec797ef2ea0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:57.318737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99931c-375e-4b03-b9fa-ea796bcafbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.319352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99931c-375e-4b03-b9fa-ea796bcafbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.319627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99931c-375e-4b03-b9fa-ea796bcafbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.319895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99931c-375e-4b03-b9fa-ea796bcafbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.321630 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa8ec1-97a1-48b3-973f-1ec797ef2ea0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.328173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19e3e2-d2e8-475e-aabe-4fcae995d13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.329059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19e3e2-d2e8-475e-aabe-4fcae995d13a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.329344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19e3e2-d2e8-475e-aabe-4fcae995d13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.329620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19e3e2-d2e8-475e-aabe-4fcae995d13a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.336679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19e3e2-d2e8-475e-aabe-4fcae995d13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.336792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99931c-375e-4b03-b9fa-ea796bcafbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.336876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19e3e2-d2e8-475e-aabe-4fcae995d13a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.336948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99931c-375e-4b03-b9fa-ea796bcafbf2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.345731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 972/1935] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:57.346528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99931c-375e-4b03-b9fa-ea796bcafbf2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:57.347827 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac99931c-375e-4b03-b9fa-ea796bcafbf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:57.351629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 964/1936] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:23:57.358100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19e3e2-d2e8-475e-aabe-4fcae995d13a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:57.365857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99931c-375e-4b03-b9fa-ea796bcafbf2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.372886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.373782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.374107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.374473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] PATH_INFO: `/v3/projects/0956cca1f69f4103a1e7f2aad89a23e2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.375840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0956cca1f69f4103a1e7f2aad89a23e2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.377918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.378044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19e3e2-d2e8-475e-aabe-4fcae995d13a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.384311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc231b-9e1d-4260-8ef4-9bd446c9bd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.386145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc231b-9e1d-4260-8ef4-9bd446c9bd94 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.386600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc231b-9e1d-4260-8ef4-9bd446c9bd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.387022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc231b-9e1d-4260-8ef4-9bd446c9bd94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.387739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc231b-9e1d-4260-8ef4-9bd446c9bd94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.390124 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc231b-9e1d-4260-8ef4-9bd446c9bd94 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.407940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 965/1937] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:57.410178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc231b-9e1d-4260-8ef4-9bd446c9bd94 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:57.424635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc231b-9e1d-4260-8ef4-9bd446c9bd94 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.430735 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ddb2d3-ba8d-4693-9f0e-92a4102122f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.431632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddb2d3-ba8d-4693-9f0e-92a4102122f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.434555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddb2d3-ba8d-4693-9f0e-92a4102122f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.435247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddb2d3-ba8d-4693-9f0e-92a4102122f5 admin admin] PATH_INFO: `/v3/projects/42067159183547dea04486b838f78075/users/9e1b11f558824f4cb8ed1c92b0b4cf67/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.448239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0956cca1f69f4103a1e7f2aad89a23e2', 'request_id': 'req-172b45ab-9b1e-4103-936e-319efcec073e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.455368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0956cca1f69f4103a1e7f2aad89a23e2', 'request_id': 'req-172b45ab-9b1e-4103-936e-319efcec073e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.458476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ddb2d3-ba8d-4693-9f0e-92a4102122f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42067159183547dea04486b838f78075, user_id=9e1b11f558824f4cb8ed1c92b0b4cf67, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.465167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ddb2d3-ba8d-4693-9f0e-92a4102122f5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.466030 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0956cca1f69f4103a1e7f2aad89a23e2', 'request_id': 'req-172b45ab-9b1e-4103-936e-319efcec073e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.479830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] The token cache is being invalidate because project 0956cca1f69f4103a1e7f2aad89a23e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:57.480124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.481076 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.482660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 973/1938] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:57 2022] DELETE /identity/v3/projects/0956cca1f69f4103a1e7f2aad89a23e2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:57.487238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:57.488735 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 966/1939] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:57 2022] PUT /identity/v3/projects/42067159183547dea04486b838f78075/users/9e1b11f558824f4cb8ed1c92b0b4cf67/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:57.488867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:57.502575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4097aba-0582-4e31-a2c2-89d4927c1b1b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.502894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4097aba-0582-4e31-a2c2-89d4927c1b1b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.503358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4097aba-0582-4e31-a2c2-89d4927c1b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.539554 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c4097aba-0582-4e31-a2c2-89d4927c1b1b None None] MFA Rules not processed for user `9e1b11f558824f4cb8ed1c92b0b4cf67`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:57.541173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c4097aba-0582-4e31-a2c2-89d4927c1b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:57.587067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b45ab-9b1e-4103-936e-319efcec073e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.593887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:57.628080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 967/1940] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:57.641681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4097aba-0582-4e31-a2c2-89d4927c1b1b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:57.644029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c4097aba-0582-4e31-a2c2-89d4927c1b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:57.688517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.689185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.689468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.689743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] PATH_INFO: `/v3/users/faf1d62615fd466c800a0b54e2214d4a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.690939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf1d62615fd466c800a0b54e2214d4a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.692983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.720838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:57.749716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4097aba-0582-4e31-a2c2-89d4927c1b1b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.755367 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceba219-f552-4beb-af2c-10a9501f3c03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0dfcbad22348fdaea32d64557e68a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.756029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceba219-f552-4beb-af2c-10a9501f3c03 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.756313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceba219-f552-4beb-af2c-10a9501f3c03 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.756826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceba219-f552-4beb-af2c-10a9501f3c03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.764247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf1d62615fd466c800a0b54e2214d4a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.765031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf1d62615fd466c800a0b54e2214d4a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.765667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf1d62615fd466c800a0b54e2214d4a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.770089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceba219-f552-4beb-af2c-10a9501f3c03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.771581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceba219-f552-4beb-af2c-10a9501f3c03 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.779408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf1d62615fd466c800a0b54e2214d4a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.784766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 968/1941] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:23:57.785202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceba219-f552-4beb-af2c-10a9501f3c03 service glance] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:57.786653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5ceba219-f552-4beb-af2c-10a9501f3c03 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:57.796104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a7b662-0a85-4898-8205-a222760554b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf1d62615fd466c800a0b54e2214d4a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.802142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceba219-f552-4beb-af2c-10a9501f3c03 service glance] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:57.807699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:57.808509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.808797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.809073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] PATH_INFO: `/v3/projects/0c537f4aaa5c43119ab973f688215a6e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.810446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c537f4aaa5c43119ab973f688215a6e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:57.813119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:57.828910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 974/1942] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:23:57 2022] DELETE /identity/v3/users/faf1d62615fd466c800a0b54e2214d4a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:57.890999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c537f4aaa5c43119ab973f688215a6e', 'request_id': 'req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.905771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c537f4aaa5c43119ab973f688215a6e', 'request_id': 'req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.917677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c537f4aaa5c43119ab973f688215a6e', 'request_id': 'req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.933386 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] The token cache is being invalidate because project 0c537f4aaa5c43119ab973f688215a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:57.933683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.934448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aa27b01c-fc32-4ccc-b5a3-9e1698761adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:57.936026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 969/1943] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:57 2022] DELETE /identity/v3/projects/0c537f4aaa5c43119ab973f688215a6e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:57.942735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582e252-3d81-41d0-957f-cdf72511725b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:57.943094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582e252-3d81-41d0-957f-cdf72511725b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:57.945489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582e252-3d81-41d0-957f-cdf72511725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:57.993351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1582e252-3d81-41d0-957f-cdf72511725b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:57.997816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1582e252-3d81-41d0-957f-cdf72511725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:58.248248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956230e6-9fcb-4077-906e-97951e9e8d12 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.248859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956230e6-9fcb-4077-906e-97951e9e8d12 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.249284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956230e6-9fcb-4077-906e-97951e9e8d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:58.251424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 975/1944] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:58.328674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-956230e6-9fcb-4077-906e-97951e9e8d12 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:58.330582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-956230e6-9fcb-4077-906e-97951e9e8d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:58.348423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 970/1945] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:23:58.353363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582e252-3d81-41d0-957f-cdf72511725b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:58.373308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582e252-3d81-41d0-957f-cdf72511725b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:58.381510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ddf2e-f30d-4c55-9911-43631d23330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:58.382669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ddf2e-f30d-4c55-9911-43631d23330f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.383027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ddf2e-f30d-4c55-9911-43631d23330f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.383242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ddf2e-f30d-4c55-9911-43631d23330f admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:58.384041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ddf2e-f30d-4c55-9911-43631d23330f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:58.384542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ddf2e-f30d-4c55-9911-43631d23330f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:58.386741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ddf2e-f30d-4c55-9911-43631d23330f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:58.404117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 976/1946] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:23:58.408831 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956230e6-9fcb-4077-906e-97951e9e8d12 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:58.420901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956230e6-9fcb-4077-906e-97951e9e8d12 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:58.426935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0914d3-35b5-4195-8c69-13da0cb09948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:58.427944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0914d3-35b5-4195-8c69-13da0cb09948 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.428287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0914d3-35b5-4195-8c69-13da0cb09948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.428675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0914d3-35b5-4195-8c69-13da0cb09948 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:58.429434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0914d3-35b5-4195-8c69-13da0cb09948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:58.431562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0914d3-35b5-4195-8c69-13da0cb09948 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:58.474214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 971/1947] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:58 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:58.478645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ddf2e-f30d-4c55-9911-43631d23330f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:58.492949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ddf2e-f30d-4c55-9911-43631d23330f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:58.506162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:58.604134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8f55b-cbdc-40a0-b1f0-35cd68aefdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:58.605231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8f55b-cbdc-40a0-b1f0-35cd68aefdfe admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.605824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8f55b-cbdc-40a0-b1f0-35cd68aefdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.606199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8f55b-cbdc-40a0-b1f0-35cd68aefdfe admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:58.607015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8f55b-cbdc-40a0-b1f0-35cd68aefdfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:58.611853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8f55b-cbdc-40a0-b1f0-35cd68aefdfe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:58.653355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 977/1948] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:58 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:58.658806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0914d3-35b5-4195-8c69-13da0cb09948 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:58.670975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0914d3-35b5-4195-8c69-13da0cb09948 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:58.683909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26435c34-6c39-41ff-9ecb-8e7d70ea283b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:58.685005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26435c34-6c39-41ff-9ecb-8e7d70ea283b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.690415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26435c34-6c39-41ff-9ecb-8e7d70ea283b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.691108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26435c34-6c39-41ff-9ecb-8e7d70ea283b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:58.691598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26435c34-6c39-41ff-9ecb-8e7d70ea283b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:58.705608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26435c34-6c39-41ff-9ecb-8e7d70ea283b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:58.714513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 972/1949] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:58.719429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8f55b-cbdc-40a0-b1f0-35cd68aefdfe admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:58.732502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8f55b-cbdc-40a0-b1f0-35cd68aefdfe admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:58.741639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26435c34-6c39-41ff-9ecb-8e7d70ea283b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:58.744099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-26435c34-6c39-41ff-9ecb-8e7d70ea283b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:58.746698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727c619-a938-40f0-8efa-878da1d765dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:58.747557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727c619-a938-40f0-8efa-878da1d765dc admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.747935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727c619-a938-40f0-8efa-878da1d765dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.748308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727c619-a938-40f0-8efa-878da1d765dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:58.749520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727c619-a938-40f0-8efa-878da1d765dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:58.751588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727c619-a938-40f0-8efa-878da1d765dc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:58.763284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26435c34-6c39-41ff-9ecb-8e7d70ea283b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:58.769760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:58.771157 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.772055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.772055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] PATH_INFO: `/v3/projects/2cddbe7eb64943e4bc27b926e198c828` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:58.775355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 978/1950] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:58.782680 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727c619-a938-40f0-8efa-878da1d765dc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:58.796419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727c619-a938-40f0-8efa-878da1d765dc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:58.801813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2784222-7015-4a22-843b-6a9464c79391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:58.802630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784222-7015-4a22-843b-6a9464c79391 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.802913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784222-7015-4a22-843b-6a9464c79391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.803190 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784222-7015-4a22-843b-6a9464c79391 admin admin] PATH_INFO: `/v3/projects/cdf049fe62cc442da4629b0dc9c39f32/users/4f04837170724f5f8fc88ccc17784d2c/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:58.804738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cddbe7eb64943e4bc27b926e198c828)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:58.806722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:58.856930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2784222-7015-4a22-843b-6a9464c79391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf049fe62cc442da4629b0dc9c39f32, user_id=4f04837170724f5f8fc88ccc17784d2c, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:58.864526 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2784222-7015-4a22-843b-6a9464c79391 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:58.902829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 979/1951] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:58 2022] PUT /identity/v3/projects/cdf049fe62cc442da4629b0dc9c39f32/users/4f04837170724f5f8fc88ccc17784d2c/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:23:58.916363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cddbe7eb64943e4bc27b926e198c828', 'request_id': 'req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:58.916647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc56793-bd3c-45f2-8fcb-c8fcfce9ffd1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.916647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc56793-bd3c-45f2-8fcb-c8fcfce9ffd1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.916647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc56793-bd3c-45f2-8fcb-c8fcfce9ffd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:58.934299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cddbe7eb64943e4bc27b926e198c828', 'request_id': 'req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:58.951270 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cddbe7eb64943e4bc27b926e198c828', 'request_id': 'req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:58.975166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] The token cache is being invalidate because project 2cddbe7eb64943e4bc27b926e198c828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:23:58.975461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:58.977365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5cc56793-bd3c-45f2-8fcb-c8fcfce9ffd1 None None] MFA Rules not processed for user `4f04837170724f5f8fc88ccc17784d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:58.979267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bfa307a0-fa6a-45f1-b5fd-a3ff131259ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:23:58.979736 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5cc56793-bd3c-45f2-8fcb-c8fcfce9ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:58.981352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 973/1952] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:23:58 2022] DELETE /identity/v3/projects/2cddbe7eb64943e4bc27b926e198c828 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:23:58.989806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33add78c-2ee6-461a-82c0-b5be45c290e5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:58.990107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33add78c-2ee6-461a-82c0-b5be45c290e5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:58.990397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33add78c-2ee6-461a-82c0-b5be45c290e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:59.008211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:59.026242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-33add78c-2ee6-461a-82c0-b5be45c290e5 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:59.028225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-33add78c-2ee6-461a-82c0-b5be45c290e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:59.118956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 974/1953] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:59.121222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf3a27-dc0f-43c4-9d68-2a561767c661 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:59.121720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf3a27-dc0f-43c4-9d68-2a561767c661 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:59.122578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf3a27-dc0f-43c4-9d68-2a561767c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:59.191115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-56bf3a27-dc0f-43c4-9d68-2a561767c661 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:23:59.192795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-56bf3a27-dc0f-43c4-9d68-2a561767c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:59.221961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 975/1954] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:59.235672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf3a27-dc0f-43c4-9d68-2a561767c661 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:59.257543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf3a27-dc0f-43c4-9d68-2a561767c661 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:59.272112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797590cb-3555-414e-81c6-73cc735e596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:59.273268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797590cb-3555-414e-81c6-73cc735e596b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:59.273840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797590cb-3555-414e-81c6-73cc735e596b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:59.274339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797590cb-3555-414e-81c6-73cc735e596b admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:59.275298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797590cb-3555-414e-81c6-73cc735e596b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:23:59.276440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797590cb-3555-414e-81c6-73cc735e596b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:59.278855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797590cb-3555-414e-81c6-73cc735e596b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:59.279292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 980/1955] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:23:59.294166 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc56793-bd3c-45f2-8fcb-c8fcfce9ffd1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:59.308001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5cc56793-bd3c-45f2-8fcb-c8fcfce9ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:23:59.390985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 976/1956] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:23:59.393583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797590cb-3555-414e-81c6-73cc735e596b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:59.436528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797590cb-3555-414e-81c6-73cc735e596b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:59.445449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731e03-6b30-4957-b974-9bb1fc1f7a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:59.451173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731e03-6b30-4957-b974-9bb1fc1f7a07 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:59.452090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731e03-6b30-4957-b974-9bb1fc1f7a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:59.452622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8731e03-6b30-4957-b974-9bb1fc1f7a07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:59.453555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8731e03-6b30-4957-b974-9bb1fc1f7a07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:59.455892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8731e03-6b30-4957-b974-9bb1fc1f7a07 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:59.509261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc56793-bd3c-45f2-8fcb-c8fcfce9ffd1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:59.515481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9b17-ca04-454b-a88e-0cb6e2b659d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:59.516630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9b17-ca04-454b-a88e-0cb6e2b659d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:59.517135 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9b17-ca04-454b-a88e-0cb6e2b659d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:59.517617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9b17-ca04-454b-a88e-0cb6e2b659d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:59.523053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 977/1957] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:23:59 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:23:59.528049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731e03-6b30-4957-b974-9bb1fc1f7a07 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:59.546228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9b17-ca04-454b-a88e-0cb6e2b659d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:59.549572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9b17-ca04-454b-a88e-0cb6e2b659d2 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:59.558593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:23:59.575107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 981/1958] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:23:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:23:59.727681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8731e03-6b30-4957-b974-9bb1fc1f7a07 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:59.739585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3f6f4-6173-430d-8057-a0d75669c2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:59.740221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3f6f4-6173-430d-8057-a0d75669c2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:59.740531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3f6f4-6173-430d-8057-a0d75669c2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:59.740807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3f6f4-6173-430d-8057-a0d75669c2c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:59.741449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3f6f4-6173-430d-8057-a0d75669c2c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:59.754726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3f6f4-6173-430d-8057-a0d75669c2c3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:59.812249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 978/1959] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:23:59 2022] POST /identity/v3/users => generated 291 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:23:59.816425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9b17-ca04-454b-a88e-0cb6e2b659d2 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:59.832273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9b17-ca04-454b-a88e-0cb6e2b659d2 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:59.846678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec900e8-8083-47e4-96e5-db0b32977687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:59.847702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec900e8-8083-47e4-96e5-db0b32977687 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:59.848048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec900e8-8083-47e4-96e5-db0b32977687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:59.848415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec900e8-8083-47e4-96e5-db0b32977687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:59.849234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec900e8-8083-47e4-96e5-db0b32977687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:59.850934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec900e8-8083-47e4-96e5-db0b32977687 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:59.866999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 982/1960] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:23:59.871246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3f6f4-6173-430d-8057-a0d75669c2c3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:59.884122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3f6f4-6173-430d-8057-a0d75669c2c3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:59.890612 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693896f5-5772-4a30-a419-f4817cbf5895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:59.891487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693896f5-5772-4a30-a419-f4817cbf5895 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:59.891771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693896f5-5772-4a30-a419-f4817cbf5895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:59.892246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693896f5-5772-4a30-a419-f4817cbf5895 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:59.892974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693896f5-5772-4a30-a419-f4817cbf5895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:23:59.894839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693896f5-5772-4a30-a419-f4817cbf5895 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:23:59.915366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 979/1961] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:23:59.921648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec900e8-8083-47e4-96e5-db0b32977687 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:23:59.935705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec900e8-8083-47e4-96e5-db0b32977687 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:23:59.954694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43589f2-d5cb-44a1-b4b4-d7703bdb97ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:23:59.955910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43589f2-d5cb-44a1-b4b4-d7703bdb97ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:23:59.956521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43589f2-d5cb-44a1-b4b4-d7703bdb97ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:23:59.957052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43589f2-d5cb-44a1-b4b4-d7703bdb97ac admin admin] PATH_INFO: `/v3/projects/e02551e2d4e74c2d977cb9fc0987ddba/users/800f79f963e74a7785289e47a2486f9e/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:23:59.996966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43589f2-d5cb-44a1-b4b4-d7703bdb97ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02551e2d4e74c2d977cb9fc0987ddba, user_id=800f79f963e74a7785289e47a2486f9e, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:00.001548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43589f2-d5cb-44a1-b4b4-d7703bdb97ac admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:00.037193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 983/1962] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:23:59 2022] PUT /identity/v3/projects/e02551e2d4e74c2d977cb9fc0987ddba/users/800f79f963e74a7785289e47a2486f9e/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:00.049378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90e33-eb5e-4e54-87d6-825d5fe2c049 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:00.049677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90e33-eb5e-4e54-87d6-825d5fe2c049 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:00.049959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90e33-eb5e-4e54-87d6-825d5fe2c049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:00.098563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-cde90e33-eb5e-4e54-87d6-825d5fe2c049 None None] MFA Rules not processed for user `800f79f963e74a7785289e47a2486f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:00.100606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cde90e33-eb5e-4e54-87d6-825d5fe2c049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:00.189278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:00.266880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 980/1963] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:00.271164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43589f2-d5cb-44a1-b4b4-d7703bdb97ac admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:00.364089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43589f2-d5cb-44a1-b4b4-d7703bdb97ac admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:00.368743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa066c2-07cc-4392-ae75-4593c746937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:00.369364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa066c2-07cc-4392-ae75-4593c746937d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:00.370321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa066c2-07cc-4392-ae75-4593c746937d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:00.370671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa066c2-07cc-4392-ae75-4593c746937d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:00.371305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa066c2-07cc-4392-ae75-4593c746937d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:00.373285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa066c2-07cc-4392-ae75-4593c746937d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:00.406623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 984/1964] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:00 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:00.410535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde90e33-eb5e-4e54-87d6-825d5fe2c049 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:00.422661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde90e33-eb5e-4e54-87d6-825d5fe2c049 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:00.484307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fb784-8972-4650-84cf-fd4501212e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:00.485043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fb784-8972-4650-84cf-fd4501212e5c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:00.485359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fb784-8972-4650-84cf-fd4501212e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:00.485635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fb784-8972-4650-84cf-fd4501212e5c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:00.486313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fb784-8972-4650-84cf-fd4501212e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:00.488447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fb784-8972-4650-84cf-fd4501212e5c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:00.512247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 981/1965] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:00 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:00.517402 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa066c2-07cc-4392-ae75-4593c746937d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:00.529638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa066c2-07cc-4392-ae75-4593c746937d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:00.534928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746f658-f666-4d2a-8de3-02698261fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:00.535598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746f658-f666-4d2a-8de3-02698261fb77 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:00.535881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746f658-f666-4d2a-8de3-02698261fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:00.536158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746f658-f666-4d2a-8de3-02698261fb77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:00.536855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746f658-f666-4d2a-8de3-02698261fb77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:00.538484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746f658-f666-4d2a-8de3-02698261fb77 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:00.550533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 985/1966] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:00.554882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fb784-8972-4650-84cf-fd4501212e5c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:00.566480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fb784-8972-4650-84cf-fd4501212e5c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:00.571287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377f7e5-709c-49c4-b416-0fd85ab191ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:00.571951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377f7e5-709c-49c4-b416-0fd85ab191ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:00.572231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377f7e5-709c-49c4-b416-0fd85ab191ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:00.572658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377f7e5-709c-49c4-b416-0fd85ab191ff admin admin] PATH_INFO: `/v3/projects/fa65dce3dec04700ba78cdb555b09dfb/users/aeada4f3aa804894b74de2611d15a575/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:00.594404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377f7e5-709c-49c4-b416-0fd85ab191ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa65dce3dec04700ba78cdb555b09dfb, user_id=aeada4f3aa804894b74de2611d15a575, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:00.594404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377f7e5-709c-49c4-b416-0fd85ab191ff admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:00.607854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 982/1967] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:00 2022] PUT /identity/v3/projects/fa65dce3dec04700ba78cdb555b09dfb/users/aeada4f3aa804894b74de2611d15a575/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:00.612040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746f658-f666-4d2a-8de3-02698261fb77 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:00.626385 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746f658-f666-4d2a-8de3-02698261fb77 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:00.708656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c9e1a-c7d7-4fcd-ab65-5c02a94affb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:00.713718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c9e1a-c7d7-4fcd-ab65-5c02a94affb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:00.713990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c9e1a-c7d7-4fcd-ab65-5c02a94affb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:00.714344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c9e1a-c7d7-4fcd-ab65-5c02a94affb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:00.715427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c9e1a-c7d7-4fcd-ab65-5c02a94affb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:00.717177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c9e1a-c7d7-4fcd-ab65-5c02a94affb0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:00.728401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 986/1968] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:00.734708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377f7e5-709c-49c4-b416-0fd85ab191ff admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:00.746924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377f7e5-709c-49c4-b416-0fd85ab191ff admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:00.752827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668bc97-80b9-4d5a-9056-1d9f045035c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:00.753585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668bc97-80b9-4d5a-9056-1d9f045035c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:00.753907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668bc97-80b9-4d5a-9056-1d9f045035c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:00.754215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668bc97-80b9-4d5a-9056-1d9f045035c6 admin admin] PATH_INFO: `/v3/projects/fa65dce3dec04700ba78cdb555b09dfb/users/aeada4f3aa804894b74de2611d15a575/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:00.756478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668bc97-80b9-4d5a-9056-1d9f045035c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa65dce3dec04700ba78cdb555b09dfb, user_id=aeada4f3aa804894b74de2611d15a575, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:00.759167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668bc97-80b9-4d5a-9056-1d9f045035c6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:00.778615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 983/1969] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:00 2022] PUT /identity/v3/projects/fa65dce3dec04700ba78cdb555b09dfb/users/aeada4f3aa804894b74de2611d15a575/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:24:00.792665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7adbc-cc89-46b4-8c51-b08d53bf51d3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:00.792963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7adbc-cc89-46b4-8c51-b08d53bf51d3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:00.793360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a7adbc-cc89-46b4-8c51-b08d53bf51d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:00.836650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a2a7adbc-cc89-46b4-8c51-b08d53bf51d3 None None] MFA Rules not processed for user `aeada4f3aa804894b74de2611d15a575`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:00.839180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a2a7adbc-cc89-46b4-8c51-b08d53bf51d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:00.882288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:00.937813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 987/1970] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:00.948986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668bc97-80b9-4d5a-9056-1d9f045035c6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:01.034352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668bc97-80b9-4d5a-9056-1d9f045035c6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:01.038297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df87960-8d3b-4f43-bca2-726de50e63bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:01.039888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df87960-8d3b-4f43-bca2-726de50e63bb service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:01.039888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df87960-8d3b-4f43-bca2-726de50e63bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:01.039888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df87960-8d3b-4f43-bca2-726de50e63bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:01.053731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df87960-8d3b-4f43-bca2-726de50e63bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:01.055229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df87960-8d3b-4f43-bca2-726de50e63bb service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:01.066994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 984/1971] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:01.191282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7adbc-cc89-46b4-8c51-b08d53bf51d3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:01.202985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a7adbc-cc89-46b4-8c51-b08d53bf51d3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:01.210994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4d5aa-bc39-4a06-93dc-29c048c6328b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:01.211768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4d5aa-bc39-4a06-93dc-29c048c6328b service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:01.212055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4d5aa-bc39-4a06-93dc-29c048c6328b service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:01.212352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4d5aa-bc39-4a06-93dc-29c048c6328b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:01.309626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4d5aa-bc39-4a06-93dc-29c048c6328b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:01.314066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4d5aa-bc39-4a06-93dc-29c048c6328b service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:01.332900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 988/1972] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:04.517527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df87960-8d3b-4f43-bca2-726de50e63bb service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:04.560220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:04.636472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df87960-8d3b-4f43-bca2-726de50e63bb service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:04.648950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:04.649865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:04.650291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:04.650947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] PATH_INFO: `/v3/users/800f79f963e74a7785289e47a2486f9e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:04.652924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=800f79f963e74a7785289e47a2486f9e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:04.655298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:04.729894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '800f79f963e74a7785289e47a2486f9e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:04.746692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '800f79f963e74a7785289e47a2486f9e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:04.747902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '800f79f963e74a7785289e47a2486f9e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:04.766055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '800f79f963e74a7785289e47a2486f9e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:04.820127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '800f79f963e74a7785289e47a2486f9e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:04.832987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 985/1973] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:04 2022] DELETE /identity/v3/users/800f79f963e74a7785289e47a2486f9e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:04.841658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d197768-74be-4d22-9ae5-70e23e8af178 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:04.841658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d197768-74be-4d22-9ae5-70e23e8af178 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:04.841658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d197768-74be-4d22-9ae5-70e23e8af178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:04.894851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4d197768-74be-4d22-9ae5-70e23e8af178 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:04.896922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4d197768-74be-4d22-9ae5-70e23e8af178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:04.991363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 989/1974] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:05.035241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:05.038361 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:05.134839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34402c-9c58-4bae-95bd-145779d93651 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:05.145294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35d005-886c-4bc0-b3b8-528dc2b18a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:05.145897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35d005-886c-4bc0-b3b8-528dc2b18a0d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:05.146235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35d005-886c-4bc0-b3b8-528dc2b18a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:05.146560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35d005-886c-4bc0-b3b8-528dc2b18a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:05.157820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35d005-886c-4bc0-b3b8-528dc2b18a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:05.159465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35d005-886c-4bc0-b3b8-528dc2b18a0d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:05.175433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 986/1975] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:06.110159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d197768-74be-4d22-9ae5-70e23e8af178 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:06.111814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4d197768-74be-4d22-9ae5-70e23e8af178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:06.134676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d197768-74be-4d22-9ae5-70e23e8af178 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:06.146362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:06.147293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:06.147565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:06.147837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] PATH_INFO: `/v3/projects/e02551e2d4e74c2d977cb9fc0987ddba` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:06.149077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02551e2d4e74c2d977cb9fc0987ddba)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:06.151801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:06.257836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02551e2d4e74c2d977cb9fc0987ddba', 'request_id': 'req-e5518b60-b63f-4dc7-9a53-8fe359a658e9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.272147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:06.273645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02551e2d4e74c2d977cb9fc0987ddba', 'request_id': 'req-e5518b60-b63f-4dc7-9a53-8fe359a658e9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.289456 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02551e2d4e74c2d977cb9fc0987ddba', 'request_id': 'req-e5518b60-b63f-4dc7-9a53-8fe359a658e9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.327567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] The token cache is being invalidate because project e02551e2d4e74c2d977cb9fc0987ddba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:06.327567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.341552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.350435 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 990/1976] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:06 2022] DELETE /identity/v3/projects/e02551e2d4e74c2d977cb9fc0987ddba => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:06.352208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35d005-886c-4bc0-b3b8-528dc2b18a0d service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:06.377287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35d005-886c-4bc0-b3b8-528dc2b18a0d service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:06.432720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:06.459817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:06.461327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:06.461628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:06.461988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] PATH_INFO: `/v3/users/aeada4f3aa804894b74de2611d15a575` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:06.464498 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeada4f3aa804894b74de2611d15a575)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:06.467547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:06.568247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeada4f3aa804894b74de2611d15a575'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.584520 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeada4f3aa804894b74de2611d15a575'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.587757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeada4f3aa804894b74de2611d15a575'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.608641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeada4f3aa804894b74de2611d15a575'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.620699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-508dc63a-db20-4133-968e-5b1aa3c253a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeada4f3aa804894b74de2611d15a575'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:06.632461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 987/1977] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:06 2022] DELETE /identity/v3/users/aeada4f3aa804894b74de2611d15a575 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:07.103936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:07.106689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:07.125454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5518b60-b63f-4dc7-9a53-8fe359a658e9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:07.226552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:07.227200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:07.227478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:07.227756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] PATH_INFO: `/v3/projects/fa65dce3dec04700ba78cdb555b09dfb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:07.229094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa65dce3dec04700ba78cdb555b09dfb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:07.231296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:07.307862 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa65dce3dec04700ba78cdb555b09dfb', 'request_id': 'req-286daf17-6a82-4e37-a9cb-6b208a1a14a1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:07.315475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa65dce3dec04700ba78cdb555b09dfb', 'request_id': 'req-286daf17-6a82-4e37-a9cb-6b208a1a14a1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:07.328718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa65dce3dec04700ba78cdb555b09dfb', 'request_id': 'req-286daf17-6a82-4e37-a9cb-6b208a1a14a1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:07.348783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] The token cache is being invalidate because project fa65dce3dec04700ba78cdb555b09dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:07.349599 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:07.350620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-286daf17-6a82-4e37-a9cb-6b208a1a14a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:07.358100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 991/1978] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:07 2022] DELETE /identity/v3/projects/fa65dce3dec04700ba78cdb555b09dfb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:07.359956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841d60b-04a6-44e8-adc5-600153e4f5f5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:07.360475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841d60b-04a6-44e8-adc5-600153e4f5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:07.360672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841d60b-04a6-44e8-adc5-600153e4f5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:07.389127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9841d60b-04a6-44e8-adc5-600153e4f5f5 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:07.396932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9841d60b-04a6-44e8-adc5-600153e4f5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:07.616541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 988/1979] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:07.631940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce506c-438f-47fe-a1e3-9efacd782471 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:07.631940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce506c-438f-47fe-a1e3-9efacd782471 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:07.631940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce506c-438f-47fe-a1e3-9efacd782471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:07.663118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0cce506c-438f-47fe-a1e3-9efacd782471 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:07.664339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0cce506c-438f-47fe-a1e3-9efacd782471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:07.679015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 992/1980] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:07.683751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841d60b-04a6-44e8-adc5-600153e4f5f5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:07.696743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841d60b-04a6-44e8-adc5-600153e4f5f5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:07.705402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c51b38-6e02-4b1d-bef3-cf5fe57c369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:07.706321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c51b38-6e02-4b1d-bef3-cf5fe57c369c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:07.706609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c51b38-6e02-4b1d-bef3-cf5fe57c369c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:07.706883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c51b38-6e02-4b1d-bef3-cf5fe57c369c admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:07.707737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c51b38-6e02-4b1d-bef3-cf5fe57c369c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:24:07.708042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c51b38-6e02-4b1d-bef3-cf5fe57c369c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:07.709875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c51b38-6e02-4b1d-bef3-cf5fe57c369c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:07.729509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 989/1981] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:24:07.734254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce506c-438f-47fe-a1e3-9efacd782471 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:07.752126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce506c-438f-47fe-a1e3-9efacd782471 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:07.758254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ba2c1-7a46-4b48-8864-c0467d4a389c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:07.758883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ba2c1-7a46-4b48-8864-c0467d4a389c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:07.759158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ba2c1-7a46-4b48-8864-c0467d4a389c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:07.759426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ba2c1-7a46-4b48-8864-c0467d4a389c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:07.760134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ba2c1-7a46-4b48-8864-c0467d4a389c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:07.762113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ba2c1-7a46-4b48-8864-c0467d4a389c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:07.809754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 993/1982] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:07 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:07.813929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c51b38-6e02-4b1d-bef3-cf5fe57c369c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:07.825849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c51b38-6e02-4b1d-bef3-cf5fe57c369c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:07.895267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:07.909432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0541d2-68c3-49f4-9221-031eb582f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:07.910233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0541d2-68c3-49f4-9221-031eb582f9fc admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:07.910749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0541d2-68c3-49f4-9221-031eb582f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:07.911294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0541d2-68c3-49f4-9221-031eb582f9fc admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:07.913874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0541d2-68c3-49f4-9221-031eb582f9fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:07.914040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0541d2-68c3-49f4-9221-031eb582f9fc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:07.956987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 990/1983] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:07 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:07.961029 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ba2c1-7a46-4b48-8864-c0467d4a389c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:07.975707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ba2c1-7a46-4b48-8864-c0467d4a389c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:08.005727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce3cb-d2a4-446d-a8c6-2a627646592e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:08.006357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce3cb-d2a4-446d-a8c6-2a627646592e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.006636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce3cb-d2a4-446d-a8c6-2a627646592e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.006906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce3cb-d2a4-446d-a8c6-2a627646592e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.007569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce3cb-d2a4-446d-a8c6-2a627646592e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:08.009270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce3cb-d2a4-446d-a8c6-2a627646592e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:08.022558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 994/1984] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:08.027575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0541d2-68c3-49f4-9221-031eb582f9fc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:08.053380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0541d2-68c3-49f4-9221-031eb582f9fc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:08.079394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178dd356-132b-44ae-8fa5-a8ece53d5633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:08.080494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dd356-132b-44ae-8fa5-a8ece53d5633 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.081205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dd356-132b-44ae-8fa5-a8ece53d5633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.081796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178dd356-132b-44ae-8fa5-a8ece53d5633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.082774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dd356-132b-44ae-8fa5-a8ece53d5633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:08.085138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178dd356-132b-44ae-8fa5-a8ece53d5633 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:08.102488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 991/1985] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:08.107795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce3cb-d2a4-446d-a8c6-2a627646592e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:08.123075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce3cb-d2a4-446d-a8c6-2a627646592e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:08.143361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1c5a9-ee05-4503-b4df-fc1c74319e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:08.143361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1c5a9-ee05-4503-b4df-fc1c74319e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.143361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1c5a9-ee05-4503-b4df-fc1c74319e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.143361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1c5a9-ee05-4503-b4df-fc1c74319e41 admin admin] PATH_INFO: `/v3/projects/258498772d2748aba1c810acbb945261/users/dd7e15344df84a68ba444bc280e44073/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.158619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1c5a9-ee05-4503-b4df-fc1c74319e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=258498772d2748aba1c810acbb945261, user_id=dd7e15344df84a68ba444bc280e44073, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:08.161446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1c5a9-ee05-4503-b4df-fc1c74319e41 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:08.176995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 995/1986] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:08 2022] PUT /identity/v3/projects/258498772d2748aba1c810acbb945261/users/dd7e15344df84a68ba444bc280e44073/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:08.192019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627ecb2-680b-434e-96a5-f05b5e8f318f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.192530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627ecb2-680b-434e-96a5-f05b5e8f318f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.192638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627ecb2-680b-434e-96a5-f05b5e8f318f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.227117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0627ecb2-680b-434e-96a5-f05b5e8f318f None None] MFA Rules not processed for user `dd7e15344df84a68ba444bc280e44073`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:08.228431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0627ecb2-680b-434e-96a5-f05b5e8f318f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:08.323811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 992/1987] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:08.328461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1c5a9-ee05-4503-b4df-fc1c74319e41 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:08.349401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:08.411653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1c5a9-ee05-4503-b4df-fc1c74319e41 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:08.423303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b8cbe-4b33-4f7c-b190-f64ad11c1845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:08.424498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b8cbe-4b33-4f7c-b190-f64ad11c1845 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.425143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b8cbe-4b33-4f7c-b190-f64ad11c1845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.425630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b8cbe-4b33-4f7c-b190-f64ad11c1845 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.426563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b8cbe-4b33-4f7c-b190-f64ad11c1845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:08.429539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b8cbe-4b33-4f7c-b190-f64ad11c1845 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:08.464901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 996/1988] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:08 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:08.466539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627ecb2-680b-434e-96a5-f05b5e8f318f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:08.477412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627ecb2-680b-434e-96a5-f05b5e8f318f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:08.554915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d45a9d-1cca-46df-895e-99427644fdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:08.555638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d45a9d-1cca-46df-895e-99427644fdbe admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.555922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d45a9d-1cca-46df-895e-99427644fdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.556293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d45a9d-1cca-46df-895e-99427644fdbe admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.557140 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d45a9d-1cca-46df-895e-99427644fdbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:08.559523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d45a9d-1cca-46df-895e-99427644fdbe admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:08.582528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 993/1989] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:08 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:08.586611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b8cbe-4b33-4f7c-b190-f64ad11c1845 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:08.598277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b8cbe-4b33-4f7c-b190-f64ad11c1845 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:08.603072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a20870-6931-41a2-af69-1c9828336d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:08.603703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a20870-6931-41a2-af69-1c9828336d32 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.603986 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a20870-6931-41a2-af69-1c9828336d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.604371 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a20870-6931-41a2-af69-1c9828336d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.605192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a20870-6931-41a2-af69-1c9828336d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:08.606759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a20870-6931-41a2-af69-1c9828336d32 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:08.618759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 997/1990] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:08.623798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d45a9d-1cca-46df-895e-99427644fdbe admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:08.634620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d45a9d-1cca-46df-895e-99427644fdbe admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:08.640973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632f3a0-4bdf-4c44-bcf4-631f48934571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:08.641582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632f3a0-4bdf-4c44-bcf4-631f48934571 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.641931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632f3a0-4bdf-4c44-bcf4-631f48934571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.642273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632f3a0-4bdf-4c44-bcf4-631f48934571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.642994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b632f3a0-4bdf-4c44-bcf4-631f48934571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:08.644725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b632f3a0-4bdf-4c44-bcf4-631f48934571 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:08.660336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 994/1991] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:24:08.664902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a20870-6931-41a2-af69-1c9828336d32 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:08.677153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a20870-6931-41a2-af69-1c9828336d32 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:08.682393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9892f2-0ae2-42ea-9429-b3ae95452018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:08.683096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9892f2-0ae2-42ea-9429-b3ae95452018 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.683365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9892f2-0ae2-42ea-9429-b3ae95452018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.684027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9892f2-0ae2-42ea-9429-b3ae95452018 admin admin] PATH_INFO: `/v3/projects/ec6fb88b34574a04b6c48979f84a00f0/users/5d1a63e7c0db410197e93323c3224c7c/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.701898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9892f2-0ae2-42ea-9429-b3ae95452018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6fb88b34574a04b6c48979f84a00f0, user_id=5d1a63e7c0db410197e93323c3224c7c, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:08.704729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9892f2-0ae2-42ea-9429-b3ae95452018 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:08.725395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 998/1992] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:08 2022] PUT /identity/v3/projects/ec6fb88b34574a04b6c48979f84a00f0/users/5d1a63e7c0db410197e93323c3224c7c/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:08.735839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ba49d-9ed9-4f0b-a848-acc5f151fe25 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.736124 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ba49d-9ed9-4f0b-a848-acc5f151fe25 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.736410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ba49d-9ed9-4f0b-a848-acc5f151fe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.768640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-533ba49d-9ed9-4f0b-a848-acc5f151fe25 None None] MFA Rules not processed for user `5d1a63e7c0db410197e93323c3224c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:08.769865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-533ba49d-9ed9-4f0b-a848-acc5f151fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:08.800950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:08.839938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 995/1993] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:08.844656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9892f2-0ae2-42ea-9429-b3ae95452018 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:08.909964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9892f2-0ae2-42ea-9429-b3ae95452018 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:08.914497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272894df-e0cc-4a32-a4b5-e2ae2dbc6abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:08.915100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272894df-e0cc-4a32-a4b5-e2ae2dbc6abf admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:08.915378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272894df-e0cc-4a32-a4b5-e2ae2dbc6abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:08.915649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272894df-e0cc-4a32-a4b5-e2ae2dbc6abf admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:08.916795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272894df-e0cc-4a32-a4b5-e2ae2dbc6abf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:08.919019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272894df-e0cc-4a32-a4b5-e2ae2dbc6abf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:08.948963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 999/1994] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:08 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:08.953068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ba49d-9ed9-4f0b-a848-acc5f151fe25 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:08.967633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ba49d-9ed9-4f0b-a848-acc5f151fe25 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:09.031815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b373849-afcf-453b-915d-a673faddb7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:09.032452 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b373849-afcf-453b-915d-a673faddb7fa admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:09.032731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b373849-afcf-453b-915d-a673faddb7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:09.033002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b373849-afcf-453b-915d-a673faddb7fa admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:09.033643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b373849-afcf-453b-915d-a673faddb7fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:09.035575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b373849-afcf-453b-915d-a673faddb7fa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:09.059420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 996/1995] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:08 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:09.063531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272894df-e0cc-4a32-a4b5-e2ae2dbc6abf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:09.074688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272894df-e0cc-4a32-a4b5-e2ae2dbc6abf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:09.079797 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53021639-3a27-4a27-b129-042ddad5aacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:09.080455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53021639-3a27-4a27-b129-042ddad5aacf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:09.080743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53021639-3a27-4a27-b129-042ddad5aacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:09.081025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53021639-3a27-4a27-b129-042ddad5aacf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:09.081685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53021639-3a27-4a27-b129-042ddad5aacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:09.083710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53021639-3a27-4a27-b129-042ddad5aacf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:09.095683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1000/1996] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:24:09.100070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b373849-afcf-453b-915d-a673faddb7fa admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:09.110742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b373849-afcf-453b-915d-a673faddb7fa admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:09.125077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c6b6c-9890-4cdf-9695-9d090371bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:09.125800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c6b6c-9890-4cdf-9695-9d090371bffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:09.126077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c6b6c-9890-4cdf-9695-9d090371bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:09.126351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c6b6c-9890-4cdf-9695-9d090371bffc admin admin] PATH_INFO: `/v3/projects/a1b6e0d929c9439ba2c6bc8b11509477/users/980e8ede71d540cd92cd4f3cf4b8ce32/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:09.142945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c6b6c-9890-4cdf-9695-9d090371bffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b6e0d929c9439ba2c6bc8b11509477, user_id=980e8ede71d540cd92cd4f3cf4b8ce32, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:09.145692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c6b6c-9890-4cdf-9695-9d090371bffc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:09.162335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 997/1997] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:09 2022] PUT /identity/v3/projects/a1b6e0d929c9439ba2c6bc8b11509477/users/980e8ede71d540cd92cd4f3cf4b8ce32/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:09.167935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53021639-3a27-4a27-b129-042ddad5aacf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:09.178367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53021639-3a27-4a27-b129-042ddad5aacf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:09.211991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:09.246376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16473db5-2b7d-40cc-aa69-2ce9fdb1d5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:09.246984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16473db5-2b7d-40cc-aa69-2ce9fdb1d5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:09.247368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16473db5-2b7d-40cc-aa69-2ce9fdb1d5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:09.247806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16473db5-2b7d-40cc-aa69-2ce9fdb1d5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:09.248613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16473db5-2b7d-40cc-aa69-2ce9fdb1d5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:09.250302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16473db5-2b7d-40cc-aa69-2ce9fdb1d5a9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:09.262000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1001/1998] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:09.266120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c6b6c-9890-4cdf-9695-9d090371bffc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:09.277321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c6b6c-9890-4cdf-9695-9d090371bffc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:09.282049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c21590-f7bb-4dd4-924a-d0d9f04baab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:09.282728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c21590-f7bb-4dd4-924a-d0d9f04baab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:09.283007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c21590-f7bb-4dd4-924a-d0d9f04baab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:09.283274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c21590-f7bb-4dd4-924a-d0d9f04baab1 admin admin] PATH_INFO: `/v3/projects/a1b6e0d929c9439ba2c6bc8b11509477/users/980e8ede71d540cd92cd4f3cf4b8ce32/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:09.285302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c21590-f7bb-4dd4-924a-d0d9f04baab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b6e0d929c9439ba2c6bc8b11509477, user_id=980e8ede71d540cd92cd4f3cf4b8ce32, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:09.288254 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c21590-f7bb-4dd4-924a-d0d9f04baab1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:09.302451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 998/1999] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:09 2022] PUT /identity/v3/projects/a1b6e0d929c9439ba2c6bc8b11509477/users/980e8ede71d540cd92cd4f3cf4b8ce32/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:24:09.314208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de36675-c840-43de-82a6-ed3ac2361ba1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:09.314493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de36675-c840-43de-82a6-ed3ac2361ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:09.314767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de36675-c840-43de-82a6-ed3ac2361ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:09.344552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4de36675-c840-43de-82a6-ed3ac2361ba1 None None] MFA Rules not processed for user `980e8ede71d540cd92cd4f3cf4b8ce32`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:09.346136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4de36675-c840-43de-82a6-ed3ac2361ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:09.432954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1002/2000] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:09.444571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c21590-f7bb-4dd4-924a-d0d9f04baab1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:09.446307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-40c21590-f7bb-4dd4-924a-d0d9f04baab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:09.522771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c21590-f7bb-4dd4-924a-d0d9f04baab1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:09.528173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c8295-0146-45c4-914b-0ada4ce6b4ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:09.528960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8295-0146-45c4-914b-0ada4ce6b4ec service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:09.529313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8295-0146-45c4-914b-0ada4ce6b4ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:09.529685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8295-0146-45c4-914b-0ada4ce6b4ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:09.579422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:09.605945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c8295-0146-45c4-914b-0ada4ce6b4ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:09.607408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c8295-0146-45c4-914b-0ada4ce6b4ec service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:09.616481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 999/2001] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:11.432120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de36675-c840-43de-82a6-ed3ac2361ba1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:11.434244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4de36675-c840-43de-82a6-ed3ac2361ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:11.466884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de36675-c840-43de-82a6-ed3ac2361ba1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:11.486874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b6a3b-c015-400d-8041-12a65c1bce58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:11.489360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b6a3b-c015-400d-8041-12a65c1bce58 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:11.489641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b6a3b-c015-400d-8041-12a65c1bce58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:11.490819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b6a3b-c015-400d-8041-12a65c1bce58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:11.588984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b6a3b-c015-400d-8041-12a65c1bce58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:11.591410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b6a3b-c015-400d-8041-12a65c1bce58 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:11.612521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1003/2002] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:12.526764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c8295-0146-45c4-914b-0ada4ce6b4ec service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:12.537619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c8295-0146-45c4-914b-0ada4ce6b4ec service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:12.541953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d6985-bf63-4dca-a4d9-b93bd644eba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:12.542904 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d6985-bf63-4dca-a4d9-b93bd644eba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:12.543178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d6985-bf63-4dca-a4d9-b93bd644eba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:12.543448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d6985-bf63-4dca-a4d9-b93bd644eba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:12.563275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d6985-bf63-4dca-a4d9-b93bd644eba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:12.567524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d6985-bf63-4dca-a4d9-b93bd644eba4 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:12.583200 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1000/2003] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:14.306104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b6a3b-c015-400d-8041-12a65c1bce58 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:14.307963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc3b6a3b-c015-400d-8041-12a65c1bce58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:14.379386 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:14.393942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b6a3b-c015-400d-8041-12a65c1bce58 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:14.398465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:14.399105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:14.399379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:14.399648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] PATH_INFO: `/v3/users/4f04837170724f5f8fc88ccc17784d2c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:14.401706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f04837170724f5f8fc88ccc17784d2c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:14.403901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:14.456186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f04837170724f5f8fc88ccc17784d2c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:14.457011 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f04837170724f5f8fc88ccc17784d2c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:14.457632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f04837170724f5f8fc88ccc17784d2c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:14.465692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f04837170724f5f8fc88ccc17784d2c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:14.478254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f04837170724f5f8fc88ccc17784d2c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:14.492193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1004/2004] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:14 2022] DELETE /identity/v3/users/4f04837170724f5f8fc88ccc17784d2c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:14.498044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557560a-1b31-4181-8121-82a2ff57566e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:14.498595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557560a-1b31-4181-8121-82a2ff57566e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:14.498595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557560a-1b31-4181-8121-82a2ff57566e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:14.518598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3557560a-1b31-4181-8121-82a2ff57566e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:14.519881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3557560a-1b31-4181-8121-82a2ff57566e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:14.588140 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1001/2005] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:24:14.598539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:14.599941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:14.674083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769d332-37b7-48b7-92b3-c67c987a710d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:14.682777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de970177-6238-4d6b-abcb-bba7856c4888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:14.683812 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de970177-6238-4d6b-abcb-bba7856c4888 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:14.684088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de970177-6238-4d6b-abcb-bba7856c4888 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:14.684360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de970177-6238-4d6b-abcb-bba7856c4888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:14.694399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de970177-6238-4d6b-abcb-bba7856c4888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:14.695865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de970177-6238-4d6b-abcb-bba7856c4888 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:14.709364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1005/2006] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:15.021061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557560a-1b31-4181-8121-82a2ff57566e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:15.022731 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3557560a-1b31-4181-8121-82a2ff57566e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:15.047202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557560a-1b31-4181-8121-82a2ff57566e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:15.066288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:15.067110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:15.067609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:15.067893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] PATH_INFO: `/v3/projects/cdf049fe62cc442da4629b0dc9c39f32` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:15.069104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf049fe62cc442da4629b0dc9c39f32)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:15.071183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:15.138480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf049fe62cc442da4629b0dc9c39f32', 'request_id': 'req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:15.145612 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf049fe62cc442da4629b0dc9c39f32', 'request_id': 'req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:15.164888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf049fe62cc442da4629b0dc9c39f32', 'request_id': 'req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:15.184125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] The token cache is being invalidate because project cdf049fe62cc442da4629b0dc9c39f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:15.184530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:15.191474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d593bc2-d7bd-4aa1-b4ba-74af2038d73b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:15.193219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1002/2007] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:15 2022] DELETE /identity/v3/projects/cdf049fe62cc442da4629b0dc9c39f32 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:16.792923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de970177-6238-4d6b-abcb-bba7856c4888 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:16.796271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-de970177-6238-4d6b-abcb-bba7856c4888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:16.818266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de970177-6238-4d6b-abcb-bba7856c4888 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:16.886022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:16.886748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:16.887177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:16.887470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] PATH_INFO: `/v3/users/dd7e15344df84a68ba444bc280e44073` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:16.888946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd7e15344df84a68ba444bc280e44073)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:16.890872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:16.913652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:16.951448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd7e15344df84a68ba444bc280e44073'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:16.952123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd7e15344df84a68ba444bc280e44073'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:16.952771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd7e15344df84a68ba444bc280e44073'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:16.959796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd7e15344df84a68ba444bc280e44073'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:16.973230 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd7e15344df84a68ba444bc280e44073'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:16.988245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1006/2008] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:16 2022] DELETE /identity/v3/users/dd7e15344df84a68ba444bc280e44073 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:16.993174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3b19c-226b-4608-b750-def5b769beba None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:16.993725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3b19c-226b-4608-b750-def5b769beba None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:16.993725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3b19c-226b-4608-b750-def5b769beba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:17.015156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-24d3b19c-226b-4608-b750-def5b769beba None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:17.016617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24d3b19c-226b-4608-b750-def5b769beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:17.063373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:17.088784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1003/2009] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:24:17.098952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:17.100371 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:17.159884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f17916-f5c3-448d-8e40-c9d25ea1552c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:17.163607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530dfec-68ac-400e-90ee-df8cb83fb0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:17.164265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530dfec-68ac-400e-90ee-df8cb83fb0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:17.164577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530dfec-68ac-400e-90ee-df8cb83fb0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:17.164851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530dfec-68ac-400e-90ee-df8cb83fb0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:17.175498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530dfec-68ac-400e-90ee-df8cb83fb0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:17.176990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8530dfec-68ac-400e-90ee-df8cb83fb0d9 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:17.186948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1007/2010] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:17.480448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3b19c-226b-4608-b750-def5b769beba None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:17.480913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24d3b19c-226b-4608-b750-def5b769beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:17.500697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3b19c-226b-4608-b750-def5b769beba None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:17.506699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:17.507937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:17.508473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:17.508745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] PATH_INFO: `/v3/projects/258498772d2748aba1c810acbb945261` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:17.511019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=258498772d2748aba1c810acbb945261)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:17.513346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:17.589363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258498772d2748aba1c810acbb945261', 'request_id': 'req-58588e37-e369-44e2-aea5-d616d42830f3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.594889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258498772d2748aba1c810acbb945261', 'request_id': 'req-58588e37-e369-44e2-aea5-d616d42830f3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.608354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258498772d2748aba1c810acbb945261', 'request_id': 'req-58588e37-e369-44e2-aea5-d616d42830f3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.621589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] The token cache is being invalidate because project 258498772d2748aba1c810acbb945261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:17.621884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.622574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.624116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1004/2011] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:17 2022] DELETE /identity/v3/projects/258498772d2748aba1c810acbb945261 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:17.628212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530dfec-68ac-400e-90ee-df8cb83fb0d9 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:17.638418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8530dfec-68ac-400e-90ee-df8cb83fb0d9 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:17.705935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:17.706471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:17.706818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:17.707015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] PATH_INFO: `/v3/users/5d1a63e7c0db410197e93323c3224c7c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:17.708226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1a63e7c0db410197e93323c3224c7c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:17.710304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:17.755340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:17.762878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1a63e7c0db410197e93323c3224c7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.763675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1a63e7c0db410197e93323c3224c7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.764534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1a63e7c0db410197e93323c3224c7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.772214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1a63e7c0db410197e93323c3224c7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.785397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1a63e7c0db410197e93323c3224c7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:17.801691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1008/2012] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:17 2022] DELETE /identity/v3/users/5d1a63e7c0db410197e93323c3224c7c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:18.166503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:18.168698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:18.183291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58588e37-e369-44e2-aea5-d616d42830f3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:18.246696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:18.247344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:18.247619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:18.247896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] PATH_INFO: `/v3/projects/ec6fb88b34574a04b6c48979f84a00f0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:18.249107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6fb88b34574a04b6c48979f84a00f0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:18.251036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:18.303740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6fb88b34574a04b6c48979f84a00f0', 'request_id': 'req-b78c722f-e086-4800-ae69-aa4aa5e5a705'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.311030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6fb88b34574a04b6c48979f84a00f0', 'request_id': 'req-b78c722f-e086-4800-ae69-aa4aa5e5a705'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.320197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6fb88b34574a04b6c48979f84a00f0', 'request_id': 'req-b78c722f-e086-4800-ae69-aa4aa5e5a705'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.329370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:18.333932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] The token cache is being invalidate because project ec6fb88b34574a04b6c48979f84a00f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:18.334328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.335535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.337232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1005/2013] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:18 2022] DELETE /identity/v3/projects/ec6fb88b34574a04b6c48979f84a00f0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:18.341693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:18.343162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:18.356654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a22f8-357f-4332-b309-921755ab0c30 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:18.418747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:18.419343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:18.419623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:18.419894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] PATH_INFO: `/v3/users/980e8ede71d540cd92cd4f3cf4b8ce32` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:18.421089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=980e8ede71d540cd92cd4f3cf4b8ce32)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:18.423234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:18.487798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '980e8ede71d540cd92cd4f3cf4b8ce32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.488503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '980e8ede71d540cd92cd4f3cf4b8ce32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.489124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '980e8ede71d540cd92cd4f3cf4b8ce32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.499525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '980e8ede71d540cd92cd4f3cf4b8ce32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.512249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce7eb900-b54f-4362-b518-04546c8cde56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '980e8ede71d540cd92cd4f3cf4b8ce32'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:18.526006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1009/2014] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:18 2022] DELETE /identity/v3/users/980e8ede71d540cd92cd4f3cf4b8ce32 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:18.844399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:18.845875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:18.859089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c722f-e086-4800-ae69-aa4aa5e5a705 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:18.936353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:18.937031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:18.937339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:18.937741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] PATH_INFO: `/v3/projects/a1b6e0d929c9439ba2c6bc8b11509477` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:18.938882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b6e0d929c9439ba2c6bc8b11509477)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:18.940906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:18.995874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b6e0d929c9439ba2c6bc8b11509477', 'request_id': 'req-32f36e56-6352-4d1e-9372-be108252db0a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:19.001414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b6e0d929c9439ba2c6bc8b11509477', 'request_id': 'req-32f36e56-6352-4d1e-9372-be108252db0a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:19.009323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b6e0d929c9439ba2c6bc8b11509477', 'request_id': 'req-32f36e56-6352-4d1e-9372-be108252db0a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:19.022336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] The token cache is being invalidate because project a1b6e0d929c9439ba2c6bc8b11509477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:19.022611 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:19.023275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32f36e56-6352-4d1e-9372-be108252db0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:19.025394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1006/2015] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:18 2022] DELETE /identity/v3/projects/a1b6e0d929c9439ba2c6bc8b11509477 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:19.030552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcf52b-48f6-431b-befe-262a5d2cbc02 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.031071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcf52b-48f6-431b-befe-262a5d2cbc02 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.031162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcf52b-48f6-431b-befe-262a5d2cbc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.054238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-16dcf52b-48f6-431b-befe-262a5d2cbc02 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:19.055532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-16dcf52b-48f6-431b-befe-262a5d2cbc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:19.091351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:19.134105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1010/2016] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:19.142247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b119b8c-fa9a-4d57-a315-49956e62b22f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.142535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b119b8c-fa9a-4d57-a315-49956e62b22f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.142819 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b119b8c-fa9a-4d57-a315-49956e62b22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.174345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5b119b8c-fa9a-4d57-a315-49956e62b22f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:19.175563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5b119b8c-fa9a-4d57-a315-49956e62b22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:19.185392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1007/2017] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:19.189683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcf52b-48f6-431b-befe-262a5d2cbc02 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:19.200835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcf52b-48f6-431b-befe-262a5d2cbc02 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:19.205446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e751a-883e-44f2-b249-d673dc6caf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:19.206071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e751a-883e-44f2-b249-d673dc6caf45 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.206342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e751a-883e-44f2-b249-d673dc6caf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.206611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e751a-883e-44f2-b249-d673dc6caf45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.207292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e751a-883e-44f2-b249-d673dc6caf45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:24:19.207588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e751a-883e-44f2-b249-d673dc6caf45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:19.209269 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e751a-883e-44f2-b249-d673dc6caf45 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:19.223476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1011/2018] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:24:19.227567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b119b8c-fa9a-4d57-a315-49956e62b22f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:19.237643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b119b8c-fa9a-4d57-a315-49956e62b22f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:19.242646 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de35f43-8ac6-4b60-8be3-709e6aea4e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:19.243275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de35f43-8ac6-4b60-8be3-709e6aea4e08 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.243561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de35f43-8ac6-4b60-8be3-709e6aea4e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.243835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de35f43-8ac6-4b60-8be3-709e6aea4e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.245050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de35f43-8ac6-4b60-8be3-709e6aea4e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:19.247096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de35f43-8ac6-4b60-8be3-709e6aea4e08 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:19.279425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1008/2019] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:19 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:19.285123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e751a-883e-44f2-b249-d673dc6caf45 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:19.294953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e751a-883e-44f2-b249-d673dc6caf45 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:19.363443 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223a5be-3195-4759-9997-0bd13d9236eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:19.364053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223a5be-3195-4759-9997-0bd13d9236eb admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.364352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223a5be-3195-4759-9997-0bd13d9236eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.364643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223a5be-3195-4759-9997-0bd13d9236eb admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.365359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223a5be-3195-4759-9997-0bd13d9236eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:19.367405 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223a5be-3195-4759-9997-0bd13d9236eb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:19.392757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1012/2020] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:19 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:19.407145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de35f43-8ac6-4b60-8be3-709e6aea4e08 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:19.418721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de35f43-8ac6-4b60-8be3-709e6aea4e08 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:19.423410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dbeab-746d-4270-90ae-7d2b403735ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:19.424014 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dbeab-746d-4270-90ae-7d2b403735ee admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.424350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dbeab-746d-4270-90ae-7d2b403735ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.424698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dbeab-746d-4270-90ae-7d2b403735ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.425348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dbeab-746d-4270-90ae-7d2b403735ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:19.426948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dbeab-746d-4270-90ae-7d2b403735ee admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:19.438705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1009/2021] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:19.442965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223a5be-3195-4759-9997-0bd13d9236eb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:19.454304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223a5be-3195-4759-9997-0bd13d9236eb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:19.459048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307d6fe-9ec0-47fd-9a26-df679887fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:19.459658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307d6fe-9ec0-47fd-9a26-df679887fc4c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.459936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307d6fe-9ec0-47fd-9a26-df679887fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.460203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307d6fe-9ec0-47fd-9a26-df679887fc4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.460882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307d6fe-9ec0-47fd-9a26-df679887fc4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:19.462515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307d6fe-9ec0-47fd-9a26-df679887fc4c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:19.474677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1013/2022] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:19.480190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dbeab-746d-4270-90ae-7d2b403735ee admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:19.491263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dbeab-746d-4270-90ae-7d2b403735ee admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:19.496795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8c702-d3cb-4f74-8b51-741fc243f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:19.497462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8c702-d3cb-4f74-8b51-741fc243f439 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.497743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8c702-d3cb-4f74-8b51-741fc243f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.498013 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8c702-d3cb-4f74-8b51-741fc243f439 admin admin] PATH_INFO: `/v3/projects/fc839eae8c504e298673b61d4de8b0e5/users/887eceef50b84dde975e15ac9e7da02b/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.516781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8c702-d3cb-4f74-8b51-741fc243f439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc839eae8c504e298673b61d4de8b0e5, user_id=887eceef50b84dde975e15ac9e7da02b, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:19.519812 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8c702-d3cb-4f74-8b51-741fc243f439 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:19.540308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1010/2023] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:19 2022] PUT /identity/v3/projects/fc839eae8c504e298673b61d4de8b0e5/users/887eceef50b84dde975e15ac9e7da02b/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:19.542491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0ff35-9eb4-49ff-bcd2-bfcd4bafa966 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.542783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0ff35-9eb4-49ff-bcd2-bfcd4bafa966 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.543062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0ff35-9eb4-49ff-bcd2-bfcd4bafa966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.567777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fbf0ff35-9eb4-49ff-bcd2-bfcd4bafa966 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:19.569056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fbf0ff35-9eb4-49ff-bcd2-bfcd4bafa966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:19.633463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1014/2024] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:19.644105 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8c702-d3cb-4f74-8b51-741fc243f439 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:19.645908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7ce8c702-d3cb-4f74-8b51-741fc243f439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:19.684720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:19.716581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8c702-d3cb-4f74-8b51-741fc243f439 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:19.720235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7e92b-98f4-476c-a4a6-4bca0c1b7118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:19.720879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7e92b-98f4-476c-a4a6-4bca0c1b7118 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:19.721434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7e92b-98f4-476c-a4a6-4bca0c1b7118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:19.721712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7e92b-98f4-476c-a4a6-4bca0c1b7118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:19.732304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7e92b-98f4-476c-a4a6-4bca0c1b7118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:19.733790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7e92b-98f4-476c-a4a6-4bca0c1b7118 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:19.744195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1011/2025] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:20.491719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315a1e6-2c74-4bca-b09d-13e9cdd973ee None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:20.492241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315a1e6-2c74-4bca-b09d-13e9cdd973ee None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:20.492329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315a1e6-2c74-4bca-b09d-13e9cdd973ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:20.534167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5315a1e6-2c74-4bca-b09d-13e9cdd973ee None None] MFA Rules not processed for user `887eceef50b84dde975e15ac9e7da02b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:20.538811 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5315a1e6-2c74-4bca-b09d-13e9cdd973ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:20.585870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:20.633956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1015/2026] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:20.645523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7e92b-98f4-476c-a4a6-4bca0c1b7118 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:20.647057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1ac7e92b-98f4-476c-a4a6-4bca0c1b7118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:20.748664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7e92b-98f4-476c-a4a6-4bca0c1b7118 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:20.752300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19216807-235d-45da-89a6-abaab72ebcdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:20.752941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19216807-235d-45da-89a6-abaab72ebcdd service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:20.753222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19216807-235d-45da-89a6-abaab72ebcdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:20.753500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19216807-235d-45da-89a6-abaab72ebcdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:20.763839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19216807-235d-45da-89a6-abaab72ebcdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:20.766304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19216807-235d-45da-89a6-abaab72ebcdd service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:20.782571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1012/2027] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:22.204043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f5fbc-d71b-46ff-9fc4-61cb31c5ef11 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:22.204611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f5fbc-d71b-46ff-9fc4-61cb31c5ef11 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:22.204983 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f5fbc-d71b-46ff-9fc4-61cb31c5ef11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:22.237305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-246f5fbc-d71b-46ff-9fc4-61cb31c5ef11 None None] MFA Rules not processed for user `887eceef50b84dde975e15ac9e7da02b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:22.238594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-246f5fbc-d71b-46ff-9fc4-61cb31c5ef11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:22.245581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1016/2028] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:24.308689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19216807-235d-45da-89a6-abaab72ebcdd service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:24.310947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-19216807-235d-45da-89a6-abaab72ebcdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:24.340922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19216807-235d-45da-89a6-abaab72ebcdd service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:24.354961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f5fbc-d71b-46ff-9fc4-61cb31c5ef11 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:24.357288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-246f5fbc-d71b-46ff-9fc4-61cb31c5ef11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:24.357534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:24.359415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:24.359415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:24.359415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] PATH_INFO: `/v3/users/9e1b11f558824f4cb8ed1c92b0b4cf67` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:24.361893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e1b11f558824f4cb8ed1c92b0b4cf67)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:24.364318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:24.389193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f5fbc-d71b-46ff-9fc4-61cb31c5ef11 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:24.396927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:24.397634 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:24.397958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:24.398449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] PATH_INFO: `/v3/users/cab1d186ab1c43749b2e187bc19e4e79` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:24.400062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cab1d186ab1c43749b2e187bc19e4e79)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:24.413487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:24.457054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e1b11f558824f4cb8ed1c92b0b4cf67'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.470161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e1b11f558824f4cb8ed1c92b0b4cf67'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.471291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e1b11f558824f4cb8ed1c92b0b4cf67'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.496233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cab1d186ab1c43749b2e187bc19e4e79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.497332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cab1d186ab1c43749b2e187bc19e4e79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.498097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cab1d186ab1c43749b2e187bc19e4e79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.511141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e1b11f558824f4cb8ed1c92b0b4cf67'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.511569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cab1d186ab1c43749b2e187bc19e4e79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.532463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5aacd6fe-26cf-4a0f-9d7a-fa7ea9c4eb71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e1b11f558824f4cb8ed1c92b0b4cf67'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.549369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1013/2029] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:24 2022] DELETE /identity/v3/users/9e1b11f558824f4cb8ed1c92b0b4cf67 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:24:24.551065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db7f7c-894e-40a6-85fb-cce952c7cd6b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:24.551525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db7f7c-894e-40a6-85fb-cce952c7cd6b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:24.551947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db7f7c-894e-40a6-85fb-cce952c7cd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:24.552345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1361c713-cf03-4082-9bb1-2fbe205aa987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cab1d186ab1c43749b2e187bc19e4e79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:24.575173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-89db7f7c-894e-40a6-85fb-cce952c7cd6b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:24.576695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89db7f7c-894e-40a6-85fb-cce952c7cd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:24.587051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1017/2030] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:24 2022] DELETE /identity/v3/users/cab1d186ab1c43749b2e187bc19e4e79 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:24.590851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b87bf-8d36-4875-aa5c-85ad7988d643 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:24.591348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b87bf-8d36-4875-aa5c-85ad7988d643 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:24.591898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b87bf-8d36-4875-aa5c-85ad7988d643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:24.622395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-080b87bf-8d36-4875-aa5c-85ad7988d643 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:24.623799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-080b87bf-8d36-4875-aa5c-85ad7988d643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:24.648418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:24.682663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1014/2031] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:24.694202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db7f7c-894e-40a6-85fb-cce952c7cd6b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:24.695890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89db7f7c-894e-40a6-85fb-cce952c7cd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:24.748563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1018/2032] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:24.773327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b87bf-8d36-4875-aa5c-85ad7988d643 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:24.781285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-080b87bf-8d36-4875-aa5c-85ad7988d643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:24.801289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db7f7c-894e-40a6-85fb-cce952c7cd6b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:24.837741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:24.860715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b87bf-8d36-4875-aa5c-85ad7988d643 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:24.869342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82f44e-6478-44e3-87ec-0fb4c6925479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:24.870404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82f44e-6478-44e3-87ec-0fb4c6925479 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:24.870523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6221ca-a16e-4698-8867-4f20715f94ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:24.870912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82f44e-6478-44e3-87ec-0fb4c6925479 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:24.871325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6221ca-a16e-4698-8867-4f20715f94ba service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:24.871447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82f44e-6478-44e3-87ec-0fb4c6925479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:24.871735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6221ca-a16e-4698-8867-4f20715f94ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:24.872048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6221ca-a16e-4698-8867-4f20715f94ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:24.882239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6221ca-a16e-4698-8867-4f20715f94ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:24.882664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82f44e-6478-44e3-87ec-0fb4c6925479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:24.883868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6221ca-a16e-4698-8867-4f20715f94ba service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:24.884186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82f44e-6478-44e3-87ec-0fb4c6925479 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:24.896432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1015/2033] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:24:24.896634 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1019/2034] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:24:25.403121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82f44e-6478-44e3-87ec-0fb4c6925479 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:25.404830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ed82f44e-6478-44e3-87ec-0fb4c6925479 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:25.420204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82f44e-6478-44e3-87ec-0fb4c6925479 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:25.426271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:25.427044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:25.427438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:25.427810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] PATH_INFO: `/v3/projects/42067159183547dea04486b838f78075` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:25.429160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42067159183547dea04486b838f78075)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:25.431254 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:25.516372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42067159183547dea04486b838f78075', 'request_id': 'req-ca8ed632-e841-4670-a2dd-cf7db4ca4188'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.524042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42067159183547dea04486b838f78075', 'request_id': 'req-ca8ed632-e841-4670-a2dd-cf7db4ca4188'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.533640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42067159183547dea04486b838f78075', 'request_id': 'req-ca8ed632-e841-4670-a2dd-cf7db4ca4188'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.547577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] The token cache is being invalidate because project 42067159183547dea04486b838f78075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:25.547840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.548436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.550484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1016/2035] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:25 2022] DELETE /identity/v3/projects/42067159183547dea04486b838f78075 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:25.556371 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74770b97-7b9c-4a6b-9721-0f23b74a183b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:25.557520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74770b97-7b9c-4a6b-9721-0f23b74a183b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:25.558178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74770b97-7b9c-4a6b-9721-0f23b74a183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:25.584342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:25.589869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:25.597325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-74770b97-7b9c-4a6b-9721-0f23b74a183b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:25.598774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-74770b97-7b9c-4a6b-9721-0f23b74a183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:25.639770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:25.681040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1020/2036] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:24:25.688317 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6bd73-9030-45b8-ba06-63ce23ada19c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:25.688637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6bd73-9030-45b8-ba06-63ce23ada19c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:25.688913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6bd73-9030-45b8-ba06-63ce23ada19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:25.694042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed632-e841-4670-a2dd-cf7db4ca4188 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:25.700521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:25.701165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:25.701449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:25.701724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] PATH_INFO: `/v3/projects/7e9cefbc1e18415492b1ea0801b92f8b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:25.702896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9cefbc1e18415492b1ea0801b92f8b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:25.704875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:25.710618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dfb6bd73-9030-45b8-ba06-63ce23ada19c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:25.711850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dfb6bd73-9030-45b8-ba06-63ce23ada19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:25.720625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1021/2037] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:24:25.725071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6bd73-9030-45b8-ba06-63ce23ada19c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:25.734755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6bd73-9030-45b8-ba06-63ce23ada19c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:25.740134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd8beb-3585-490b-93f2-b9514593ca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:25.740793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd8beb-3585-490b-93f2-b9514593ca15 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:25.741077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd8beb-3585-490b-93f2-b9514593ca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:25.741348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd8beb-3585-490b-93f2-b9514593ca15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:25.742001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd8beb-3585-490b-93f2-b9514593ca15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:24:25.742302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd8beb-3585-490b-93f2-b9514593ca15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:25.743931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd8beb-3585-490b-93f2-b9514593ca15 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:25.756282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1022/2038] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:24:25.761268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd8beb-3585-490b-93f2-b9514593ca15 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:25.769923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9cefbc1e18415492b1ea0801b92f8b', 'request_id': 'req-c12ccc49-f033-4673-935a-af47cfd454e6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.771570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd8beb-3585-490b-93f2-b9514593ca15 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:25.776570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9cefbc1e18415492b1ea0801b92f8b', 'request_id': 'req-c12ccc49-f033-4673-935a-af47cfd454e6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.786012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9cefbc1e18415492b1ea0801b92f8b', 'request_id': 'req-c12ccc49-f033-4673-935a-af47cfd454e6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.808635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] The token cache is being invalidate because project 7e9cefbc1e18415492b1ea0801b92f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:25.808928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.809620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:25.812041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1017/2039] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:25 2022] DELETE /identity/v3/projects/7e9cefbc1e18415492b1ea0801b92f8b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:25.816079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:25.818562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:25.836598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db00140-2358-45d4-a1a9-0e8dd9481214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:25.837907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db00140-2358-45d4-a1a9-0e8dd9481214 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:25.837907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db00140-2358-45d4-a1a9-0e8dd9481214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:25.838978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db00140-2358-45d4-a1a9-0e8dd9481214 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:25.839897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db00140-2358-45d4-a1a9-0e8dd9481214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:25.841991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db00140-2358-45d4-a1a9-0e8dd9481214 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:25.893240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1023/2040] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:25 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:25.897893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db00140-2358-45d4-a1a9-0e8dd9481214 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:25.901222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4db00140-2358-45d4-a1a9-0e8dd9481214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:25.915527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ccc49-f033-4673-935a-af47cfd454e6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:25.970520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:25.993205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:25.993874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:25.994145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db00140-2358-45d4-a1a9-0e8dd9481214 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:25.994313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:25.994574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] PATH_INFO: `/v3/users/3b12100b380b429e823ecf44f7f86ec0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:25.995871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b12100b380b429e823ecf44f7f86ec0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:25.997886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:26.000625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dee90e-126d-41f4-b9e7-9dde72ff677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:26.001261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dee90e-126d-41f4-b9e7-9dde72ff677b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:26.001705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dee90e-126d-41f4-b9e7-9dde72ff677b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:26.002012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dee90e-126d-41f4-b9e7-9dde72ff677b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:26.002665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dee90e-126d-41f4-b9e7-9dde72ff677b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:26.002981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:26.004672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dee90e-126d-41f4-b9e7-9dde72ff677b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:26.029697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1024/2041] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:25 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:26.033944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dee90e-126d-41f4-b9e7-9dde72ff677b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:26.045276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dee90e-126d-41f4-b9e7-9dde72ff677b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:26.050025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fff7f-9d6f-4815-a95f-801791363f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:26.050641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fff7f-9d6f-4815-a95f-801791363f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:26.050944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fff7f-9d6f-4815-a95f-801791363f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:26.051194 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fff7f-9d6f-4815-a95f-801791363f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:26.051846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fff7f-9d6f-4815-a95f-801791363f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:26.053131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b12100b380b429e823ecf44f7f86ec0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.053513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fff7f-9d6f-4815-a95f-801791363f7d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:26.062883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b12100b380b429e823ecf44f7f86ec0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.063884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b12100b380b429e823ecf44f7f86ec0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.064698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1025/2042] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:24:26.068668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fff7f-9d6f-4815-a95f-801791363f7d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:26.079537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b12100b380b429e823ecf44f7f86ec0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.080273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fff7f-9d6f-4815-a95f-801791363f7d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:26.087092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7de98-1560-43f2-aa94-d7118266a4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:26.087711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7de98-1560-43f2-aa94-d7118266a4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:26.088078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7de98-1560-43f2-aa94-d7118266a4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:26.088776 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7de98-1560-43f2-aa94-d7118266a4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:26.089277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7de98-1560-43f2-aa94-d7118266a4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:26.090947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7de98-1560-43f2-aa94-d7118266a4f4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:26.095080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b12100b380b429e823ecf44f7f86ec0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.107154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1018/2043] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:25 2022] DELETE /identity/v3/users/3b12100b380b429e823ecf44f7f86ec0 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:24:26.107411 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1026/2044] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:24:26.109056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:26.110546 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:26.203185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bb03f-b51c-4118-aba5-684b9345d1c9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:26.208382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f3cbb-a9dd-401e-af61-64560b4fd16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:26.209222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f3cbb-a9dd-401e-af61-64560b4fd16a admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:26.209614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f3cbb-a9dd-401e-af61-64560b4fd16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:26.209986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f3cbb-a9dd-401e-af61-64560b4fd16a admin admin] PATH_INFO: `/v3/projects/90628c1360644da79e6825261c5ae2fb/users/764459d91f474c8bac6f108700cf220f/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:26.239849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f3cbb-a9dd-401e-af61-64560b4fd16a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90628c1360644da79e6825261c5ae2fb, user_id=764459d91f474c8bac6f108700cf220f, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:26.242658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f3cbb-a9dd-401e-af61-64560b4fd16a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:26.261191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1019/2045] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:26 2022] PUT /identity/v3/projects/90628c1360644da79e6825261c5ae2fb/users/764459d91f474c8bac6f108700cf220f/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:24:26.272116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6710a39-f30b-463e-85a3-1c731d2451b6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:26.272414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6710a39-f30b-463e-85a3-1c731d2451b6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:26.272701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6710a39-f30b-463e-85a3-1c731d2451b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:26.318593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d6710a39-f30b-463e-85a3-1c731d2451b6 None None] MFA Rules not processed for user `764459d91f474c8bac6f108700cf220f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:26.321163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d6710a39-f30b-463e-85a3-1c731d2451b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:26.427573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1027/2046] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:24:26.438224 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f3cbb-a9dd-401e-af61-64560b4fd16a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:26.443896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-129f3cbb-a9dd-401e-af61-64560b4fd16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:26.476374 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6710a39-f30b-463e-85a3-1c731d2451b6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:26.480161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d6710a39-f30b-463e-85a3-1c731d2451b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:26.550886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f3cbb-a9dd-401e-af61-64560b4fd16a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:26.554909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d161217-ba61-4dfb-8777-b3ac30319d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:26.555624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d161217-ba61-4dfb-8777-b3ac30319d04 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:26.555961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d161217-ba61-4dfb-8777-b3ac30319d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:26.556299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d161217-ba61-4dfb-8777-b3ac30319d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:26.568041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d161217-ba61-4dfb-8777-b3ac30319d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:26.569628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d161217-ba61-4dfb-8777-b3ac30319d04 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:26.571925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6710a39-f30b-463e-85a3-1c731d2451b6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:26.577300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:26.578051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:26.578439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:26.578813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] PATH_INFO: `/v3/projects/3ec65f0f3fea4e498d7b4afda211efb1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:26.580142 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec65f0f3fea4e498d7b4afda211efb1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:26.581642 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1020/2047] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:26.582464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:26.668943 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:26.672364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec65f0f3fea4e498d7b4afda211efb1', 'request_id': 'req-32620cae-9458-4815-a57a-e772d8f1fbd2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.684040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec65f0f3fea4e498d7b4afda211efb1', 'request_id': 'req-32620cae-9458-4815-a57a-e772d8f1fbd2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.700860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec65f0f3fea4e498d7b4afda211efb1', 'request_id': 'req-32620cae-9458-4815-a57a-e772d8f1fbd2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.722576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] The token cache is being invalidate because project 3ec65f0f3fea4e498d7b4afda211efb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:26.723103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.724104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32620cae-9458-4815-a57a-e772d8f1fbd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:26.731027 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1028/2048] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:26 2022] DELETE /identity/v3/projects/3ec65f0f3fea4e498d7b4afda211efb1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:26.732828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b09bd9-d1b5-458f-bca1-de022e443f06 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:26.733129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b09bd9-d1b5-458f-bca1-de022e443f06 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:26.733406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b09bd9-d1b5-458f-bca1-de022e443f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:26.774312 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-89b09bd9-d1b5-458f-bca1-de022e443f06 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:26.775925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89b09bd9-d1b5-458f-bca1-de022e443f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:26.783453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:26.889307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1021/2049] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:26.896876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29b88d-3d8c-48f8-ad1b-b3eed0ec2e08 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:26.897226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29b88d-3d8c-48f8-ad1b-b3eed0ec2e08 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:26.898109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29b88d-3d8c-48f8-ad1b-b3eed0ec2e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:26.922694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2c29b88d-3d8c-48f8-ad1b-b3eed0ec2e08 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:26.923894 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2c29b88d-3d8c-48f8-ad1b-b3eed0ec2e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:26.939940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b09bd9-d1b5-458f-bca1-de022e443f06 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:26.985901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1029/2050] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:26.994192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b09bd9-d1b5-458f-bca1-de022e443f06 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.007828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97c3ea-1bd1-4e15-a875-e810181ff719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.009613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97c3ea-1bd1-4e15-a875-e810181ff719 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.012703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97c3ea-1bd1-4e15-a875-e810181ff719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.013035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97c3ea-1bd1-4e15-a875-e810181ff719 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.013790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97c3ea-1bd1-4e15-a875-e810181ff719 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:24:27.015047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97c3ea-1bd1-4e15-a875-e810181ff719 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.021267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97c3ea-1bd1-4e15-a875-e810181ff719 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:27.052342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1022/2051] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:24:27.059332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29b88d-3d8c-48f8-ad1b-b3eed0ec2e08 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:27.076749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29b88d-3d8c-48f8-ad1b-b3eed0ec2e08 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.085872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bd938-6d62-46eb-95bb-0f3f53e17c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.087022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bd938-6d62-46eb-95bb-0f3f53e17c9f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.087491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bd938-6d62-46eb-95bb-0f3f53e17c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.088430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bd938-6d62-46eb-95bb-0f3f53e17c9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.089354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bd938-6d62-46eb-95bb-0f3f53e17c9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.093124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bd938-6d62-46eb-95bb-0f3f53e17c9f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:27.151295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1030/2052] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:27 2022] POST /identity/v3/projects => generated 389 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:27.155315 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97c3ea-1bd1-4e15-a875-e810181ff719 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:27.175201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97c3ea-1bd1-4e15-a875-e810181ff719 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.294236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c0a0e-8c6e-4b12-9696-073d7561f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.294847 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0a0e-8c6e-4b12-9696-073d7561f253 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.295120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0a0e-8c6e-4b12-9696-073d7561f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.295387 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c0a0e-8c6e-4b12-9696-073d7561f253 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.296047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c0a0e-8c6e-4b12-9696-073d7561f253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.298004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c0a0e-8c6e-4b12-9696-073d7561f253 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:27.327281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1023/2053] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:27 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:27.331323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bd938-6d62-46eb-95bb-0f3f53e17c9f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:27.343414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bd938-6d62-46eb-95bb-0f3f53e17c9f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.348445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2de8-bc0b-4c4f-8917-cd619d7fd480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.349066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2de8-bc0b-4c4f-8917-cd619d7fd480 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.349347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2de8-bc0b-4c4f-8917-cd619d7fd480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.349620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2de8-bc0b-4c4f-8917-cd619d7fd480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.350314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a2de8-bc0b-4c4f-8917-cd619d7fd480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.352324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a2de8-bc0b-4c4f-8917-cd619d7fd480 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:27.363038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1031/2054] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:27.367104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c0a0e-8c6e-4b12-9696-073d7561f253 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:27.378473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c0a0e-8c6e-4b12-9696-073d7561f253 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.383783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f3cfb-85b8-4cd2-9791-506e49a44d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.384635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f3cfb-85b8-4cd2-9791-506e49a44d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.385708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f3cfb-85b8-4cd2-9791-506e49a44d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.386050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f3cfb-85b8-4cd2-9791-506e49a44d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.386720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f3cfb-85b8-4cd2-9791-506e49a44d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.388354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f3cfb-85b8-4cd2-9791-506e49a44d4d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:27.405142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1024/2055] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:27.407581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2de8-bc0b-4c4f-8917-cd619d7fd480 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:27.419665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2de8-bc0b-4c4f-8917-cd619d7fd480 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.425077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bf0d6-957e-4d77-834b-8d33f3b79cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.425856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bf0d6-957e-4d77-834b-8d33f3b79cdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.426203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bf0d6-957e-4d77-834b-8d33f3b79cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.426556 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50bf0d6-957e-4d77-834b-8d33f3b79cdf admin admin] PATH_INFO: `/v3/projects/800d326ff6d44c34af55f6bf1f22edf3/users/743a13f799694403a8c3396e6ced4d1c/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.444132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bf0d6-957e-4d77-834b-8d33f3b79cdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=800d326ff6d44c34af55f6bf1f22edf3, user_id=743a13f799694403a8c3396e6ced4d1c, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.446995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50bf0d6-957e-4d77-834b-8d33f3b79cdf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:27.464589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1032/2056] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:27 2022] PUT /identity/v3/projects/800d326ff6d44c34af55f6bf1f22edf3/users/743a13f799694403a8c3396e6ced4d1c/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:27.474988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e64df8-a5d6-48ef-9e8c-1493c0e2ee0e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.475489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e64df8-a5d6-48ef-9e8c-1493c0e2ee0e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.475564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e64df8-a5d6-48ef-9e8c-1493c0e2ee0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.505445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b2e64df8-a5d6-48ef-9e8c-1493c0e2ee0e None None] MFA Rules not processed for user `743a13f799694403a8c3396e6ced4d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:27.506679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b2e64df8-a5d6-48ef-9e8c-1493c0e2ee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:27.597588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1025/2057] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:27.602274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bf0d6-957e-4d77-834b-8d33f3b79cdf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:27.681593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50bf0d6-957e-4d77-834b-8d33f3b79cdf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.687103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7f708-56f5-4245-945c-9406ec219d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.687880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7f708-56f5-4245-945c-9406ec219d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.688172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7f708-56f5-4245-945c-9406ec219d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.688477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7f708-56f5-4245-945c-9406ec219d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.689121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7f708-56f5-4245-945c-9406ec219d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.691084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7f708-56f5-4245-945c-9406ec219d78 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:27.725624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1033/2058] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:27 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:27.731983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e64df8-a5d6-48ef-9e8c-1493c0e2ee0e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:27.745846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:27.755814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e64df8-a5d6-48ef-9e8c-1493c0e2ee0e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.862393 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5259-5886-4579-b373-c558c8b4f0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.862982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5259-5886-4579-b373-c558c8b4f0dc admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.863453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5259-5886-4579-b373-c558c8b4f0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.863864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f5259-5886-4579-b373-c558c8b4f0dc admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.865095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f5259-5886-4579-b373-c558c8b4f0dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.868899 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f5259-5886-4579-b373-c558c8b4f0dc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:27.896758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1026/2059] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:27 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:27.903226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7f708-56f5-4245-945c-9406ec219d78 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:27.915245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7f708-56f5-4245-945c-9406ec219d78 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.923221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad64e2c-7a55-411c-b71b-3eb2ce1e02c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.923903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64e2c-7a55-411c-b71b-3eb2ce1e02c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.924267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64e2c-7a55-411c-b71b-3eb2ce1e02c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.924597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64e2c-7a55-411c-b71b-3eb2ce1e02c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.925532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad64e2c-7a55-411c-b71b-3eb2ce1e02c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.927182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad64e2c-7a55-411c-b71b-3eb2ce1e02c1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:27.940676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1034/2060] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:27.944903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5259-5886-4579-b373-c558c8b4f0dc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:27.956231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f5259-5886-4579-b373-c558c8b4f0dc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:27.963996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674b596-1056-4017-9f78-a462950d2e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:27.964704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674b596-1056-4017-9f78-a462950d2e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:27.964989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674b596-1056-4017-9f78-a462950d2e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:27.965265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674b596-1056-4017-9f78-a462950d2e45 admin admin] PATH_INFO: `/v3/projects/14c8fa0469c443ca979eefb2ae2e686c/users/466dcf00fddf49f0a9f9f52167f08615/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:27.984466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674b596-1056-4017-9f78-a462950d2e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c8fa0469c443ca979eefb2ae2e686c, user_id=466dcf00fddf49f0a9f9f52167f08615, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:27.990500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674b596-1056-4017-9f78-a462950d2e45 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:28.014073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1027/2061] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:27 2022] PUT /identity/v3/projects/14c8fa0469c443ca979eefb2ae2e686c/users/466dcf00fddf49f0a9f9f52167f08615/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:28.016537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad64e2c-7a55-411c-b71b-3eb2ce1e02c1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:28.028193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad64e2c-7a55-411c-b71b-3eb2ce1e02c1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:28.076737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:28.117974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdb21b-6503-40a2-93f4-23faa7752bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:28.118684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdb21b-6503-40a2-93f4-23faa7752bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:28.119657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdb21b-6503-40a2-93f4-23faa7752bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:28.121263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdb21b-6503-40a2-93f4-23faa7752bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:28.122031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdb21b-6503-40a2-93f4-23faa7752bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:28.124283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdb21b-6503-40a2-93f4-23faa7752bbf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:28.140121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1035/2062] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:28.141374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674b596-1056-4017-9f78-a462950d2e45 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:28.153390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674b596-1056-4017-9f78-a462950d2e45 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:28.159181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4d6c-d6c6-4d0f-a643-8464435bb7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:28.159849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4d6c-d6c6-4d0f-a643-8464435bb7d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:28.160131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4d6c-d6c6-4d0f-a643-8464435bb7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:28.160435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4d6c-d6c6-4d0f-a643-8464435bb7d4 admin admin] PATH_INFO: `/v3/projects/14c8fa0469c443ca979eefb2ae2e686c/users/466dcf00fddf49f0a9f9f52167f08615/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:28.163073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a4d6c-d6c6-4d0f-a643-8464435bb7d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c8fa0469c443ca979eefb2ae2e686c, user_id=466dcf00fddf49f0a9f9f52167f08615, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:28.167645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a4d6c-d6c6-4d0f-a643-8464435bb7d4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:28.181811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1028/2063] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:28 2022] PUT /identity/v3/projects/14c8fa0469c443ca979eefb2ae2e686c/users/466dcf00fddf49f0a9f9f52167f08615/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:24:28.193391 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516cf08-25a4-436a-82e8-2659ea7ee147 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:28.193952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516cf08-25a4-436a-82e8-2659ea7ee147 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:28.194325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516cf08-25a4-436a-82e8-2659ea7ee147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:28.235178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d516cf08-25a4-436a-82e8-2659ea7ee147 None None] MFA Rules not processed for user `466dcf00fddf49f0a9f9f52167f08615`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:28.236897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d516cf08-25a4-436a-82e8-2659ea7ee147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:28.356490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1036/2064] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:28.368948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4d6c-d6c6-4d0f-a643-8464435bb7d4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:28.380655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4f0a4d6c-d6c6-4d0f-a643-8464435bb7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:28.454598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4d6c-d6c6-4d0f-a643-8464435bb7d4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:28.458469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a782a0c-f08a-483a-a646-31bb93a9664d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:28.458800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a782a0c-f08a-483a-a646-31bb93a9664d service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:28.459077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a782a0c-f08a-483a-a646-31bb93a9664d service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:28.459343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a782a0c-f08a-483a-a646-31bb93a9664d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:28.470739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a782a0c-f08a-483a-a646-31bb93a9664d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:28.470739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a782a0c-f08a-483a-a646-31bb93a9664d service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:28.480821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1029/2065] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:28.627184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516cf08-25a4-436a-82e8-2659ea7ee147 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:28.691590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516cf08-25a4-436a-82e8-2659ea7ee147 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:28.696148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:28.696816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:28.697096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:28.697506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] PATH_INFO: `/v3/users/743a13f799694403a8c3396e6ced4d1c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:28.698854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=743a13f799694403a8c3396e6ced4d1c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:28.700839 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:28.747742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743a13f799694403a8c3396e6ced4d1c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:28.748317 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743a13f799694403a8c3396e6ced4d1c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:28.748953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743a13f799694403a8c3396e6ced4d1c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:28.755754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743a13f799694403a8c3396e6ced4d1c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:28.763777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:28.769541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743a13f799694403a8c3396e6ced4d1c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:28.782193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1037/2066] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:28 2022] DELETE /identity/v3/users/743a13f799694403a8c3396e6ced4d1c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:28.786871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b5904-19dc-4fc4-836d-6474fa0386e2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:28.787156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b5904-19dc-4fc4-836d-6474fa0386e2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:28.787428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b5904-19dc-4fc4-836d-6474fa0386e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:28.807571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3b6b5904-19dc-4fc4-836d-6474fa0386e2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:28.808811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3b6b5904-19dc-4fc4-836d-6474fa0386e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:28.859264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:28.868880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1030/2067] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:28.879382 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:28.880882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:28.938163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b753b-b70e-460f-a19c-a7eaaa2d73b1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:28.941961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff79a4f-c77a-4479-b4d8-48c9b21f87dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:28.942558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff79a4f-c77a-4479-b4d8-48c9b21f87dd service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:28.942842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff79a4f-c77a-4479-b4d8-48c9b21f87dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:28.943118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff79a4f-c77a-4479-b4d8-48c9b21f87dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:28.952251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff79a4f-c77a-4479-b4d8-48c9b21f87dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:28.953804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff79a4f-c77a-4479-b4d8-48c9b21f87dd service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:28.964206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1038/2068] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:29.234647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b5904-19dc-4fc4-836d-6474fa0386e2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:29.236100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3b6b5904-19dc-4fc4-836d-6474fa0386e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:29.253007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b5904-19dc-4fc4-836d-6474fa0386e2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:29.260823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:29.261474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:29.261754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:29.262019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] PATH_INFO: `/v3/projects/800d326ff6d44c34af55f6bf1f22edf3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:29.264027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] RBAC: Authorizing `identity:delete_project(project_id=800d326ff6d44c34af55f6bf1f22edf3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:29.266643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:29.327827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800d326ff6d44c34af55f6bf1f22edf3', 'request_id': 'req-36d2a221-7316-44c1-b748-3820be93300e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.333982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800d326ff6d44c34af55f6bf1f22edf3', 'request_id': 'req-36d2a221-7316-44c1-b748-3820be93300e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.347621 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800d326ff6d44c34af55f6bf1f22edf3', 'request_id': 'req-36d2a221-7316-44c1-b748-3820be93300e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.360516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] The token cache is being invalidate because project 800d326ff6d44c34af55f6bf1f22edf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:29.360897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.361740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.363556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1031/2069] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:29 2022] DELETE /identity/v3/projects/800d326ff6d44c34af55f6bf1f22edf3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:29.367365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff79a4f-c77a-4479-b4d8-48c9b21f87dd service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:29.377707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff79a4f-c77a-4479-b4d8-48c9b21f87dd service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:29.456800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:29.457575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:29.457851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:29.458296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] PATH_INFO: `/v3/users/466dcf00fddf49f0a9f9f52167f08615` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:29.459558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=466dcf00fddf49f0a9f9f52167f08615)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:29.461838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:29.527977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466dcf00fddf49f0a9f9f52167f08615'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.529171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466dcf00fddf49f0a9f9f52167f08615'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.529919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466dcf00fddf49f0a9f9f52167f08615'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.537933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466dcf00fddf49f0a9f9f52167f08615'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.551803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f4507e99-d93a-462d-9290-2afe72a26fb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466dcf00fddf49f0a9f9f52167f08615'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:29.566735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1039/2070] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:29 2022] DELETE /identity/v3/users/466dcf00fddf49f0a9f9f52167f08615 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:29.888149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:29.889648 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:29.913706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d2a221-7316-44c1-b748-3820be93300e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:29.970512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:29.971149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:29.971429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:29.971700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] PATH_INFO: `/v3/projects/14c8fa0469c443ca979eefb2ae2e686c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:29.972967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14c8fa0469c443ca979eefb2ae2e686c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:29.974893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:30.021875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14c8fa0469c443ca979eefb2ae2e686c', 'request_id': 'req-a709efb5-f2f5-405d-8c4b-089462fd6a27'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:30.027922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14c8fa0469c443ca979eefb2ae2e686c', 'request_id': 'req-a709efb5-f2f5-405d-8c4b-089462fd6a27'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:30.035699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14c8fa0469c443ca979eefb2ae2e686c', 'request_id': 'req-a709efb5-f2f5-405d-8c4b-089462fd6a27'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:30.053781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] The token cache is being invalidate because project 14c8fa0469c443ca979eefb2ae2e686c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:30.054065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:30.054820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a709efb5-f2f5-405d-8c4b-089462fd6a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:30.056298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1032/2071] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:29 2022] DELETE /identity/v3/projects/14c8fa0469c443ca979eefb2ae2e686c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:30.061727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4407db-0d1b-4f28-9ace-ad2dd42d55c6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.062192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4407db-0d1b-4f28-9ace-ad2dd42d55c6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.062306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4407db-0d1b-4f28-9ace-ad2dd42d55c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.076844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:30.083669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9f4407db-0d1b-4f28-9ace-ad2dd42d55c6 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:30.085351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9f4407db-0d1b-4f28-9ace-ad2dd42d55c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:30.149070 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1040/2072] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:30.157363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e80ad9-3637-42da-979f-9f500c9769a9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.157654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e80ad9-3637-42da-979f-9f500c9769a9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.157933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e80ad9-3637-42da-979f-9f500c9769a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.178318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-13e80ad9-3637-42da-979f-9f500c9769a9 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:30.179514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-13e80ad9-3637-42da-979f-9f500c9769a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:30.187961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1033/2073] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:30.192164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4407db-0d1b-4f28-9ace-ad2dd42d55c6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:30.202134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4407db-0d1b-4f28-9ace-ad2dd42d55c6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:30.206790 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31c45a-358a-4da6-8fc9-a7b564bcc45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:30.207392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31c45a-358a-4da6-8fc9-a7b564bcc45d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.207673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31c45a-358a-4da6-8fc9-a7b564bcc45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.207946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31c45a-358a-4da6-8fc9-a7b564bcc45d admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.208618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31c45a-358a-4da6-8fc9-a7b564bcc45d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:24:30.208946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31c45a-358a-4da6-8fc9-a7b564bcc45d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:30.210556 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31c45a-358a-4da6-8fc9-a7b564bcc45d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:30.225437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1041/2074] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:24:30.230611 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e80ad9-3637-42da-979f-9f500c9769a9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:30.239827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e80ad9-3637-42da-979f-9f500c9769a9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:30.245077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241509e-2b93-484c-bcd4-b5289d88fa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:30.245706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241509e-2b93-484c-bcd4-b5289d88fa77 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.245989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241509e-2b93-484c-bcd4-b5289d88fa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.246263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241509e-2b93-484c-bcd4-b5289d88fa77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.246912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241509e-2b93-484c-bcd4-b5289d88fa77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:30.248869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241509e-2b93-484c-bcd4-b5289d88fa77 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:30.253331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:30.277772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1034/2075] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:30 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:30.281973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31c45a-358a-4da6-8fc9-a7b564bcc45d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:30.292957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31c45a-358a-4da6-8fc9-a7b564bcc45d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:30.348357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292fc1f-dd84-40f0-8f74-7d48b2026336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:30.348902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292fc1f-dd84-40f0-8f74-7d48b2026336 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.349185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292fc1f-dd84-40f0-8f74-7d48b2026336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.349458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292fc1f-dd84-40f0-8f74-7d48b2026336 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.350091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292fc1f-dd84-40f0-8f74-7d48b2026336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:30.352026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292fc1f-dd84-40f0-8f74-7d48b2026336 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:30.373650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1042/2076] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:30 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:30.377728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241509e-2b93-484c-bcd4-b5289d88fa77 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:30.388054 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241509e-2b93-484c-bcd4-b5289d88fa77 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:30.392655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9949bf7-3b54-4774-8770-1c0a9d9da036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:30.393248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9949bf7-3b54-4774-8770-1c0a9d9da036 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.393527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9949bf7-3b54-4774-8770-1c0a9d9da036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.393798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9949bf7-3b54-4774-8770-1c0a9d9da036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.394443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9949bf7-3b54-4774-8770-1c0a9d9da036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:30.396080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9949bf7-3b54-4774-8770-1c0a9d9da036 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:30.407122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1035/2077] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:30.410835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292fc1f-dd84-40f0-8f74-7d48b2026336 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:30.423462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292fc1f-dd84-40f0-8f74-7d48b2026336 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:30.428666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979aa9e-a645-40fe-8720-7c1a1d69e818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:30.429364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979aa9e-a645-40fe-8720-7c1a1d69e818 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.429556 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979aa9e-a645-40fe-8720-7c1a1d69e818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.429828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979aa9e-a645-40fe-8720-7c1a1d69e818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.435750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979aa9e-a645-40fe-8720-7c1a1d69e818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:30.437621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979aa9e-a645-40fe-8720-7c1a1d69e818 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:30.449547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1043/2078] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:30.457185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9949bf7-3b54-4774-8770-1c0a9d9da036 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:30.468883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9949bf7-3b54-4774-8770-1c0a9d9da036 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:30.473542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934a7f8-beef-41e8-ae6c-c2396b40a503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:30.474165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934a7f8-beef-41e8-ae6c-c2396b40a503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.474449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934a7f8-beef-41e8-ae6c-c2396b40a503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.474723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934a7f8-beef-41e8-ae6c-c2396b40a503 admin admin] PATH_INFO: `/v3/projects/7ee621ef88e04e25a9c8aa3e54e3f98e/users/cf830af8101b4e56877e14e21255f444/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.491979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934a7f8-beef-41e8-ae6c-c2396b40a503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee621ef88e04e25a9c8aa3e54e3f98e, user_id=cf830af8101b4e56877e14e21255f444, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:30.495469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934a7f8-beef-41e8-ae6c-c2396b40a503 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:30.508362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1036/2079] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:30 2022] PUT /identity/v3/projects/7ee621ef88e04e25a9c8aa3e54e3f98e/users/cf830af8101b4e56877e14e21255f444/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:30.519315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5c34f-7ea9-42a5-9a04-b4658e8b3ca1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.519603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5c34f-7ea9-42a5-9a04-b4658e8b3ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.519965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5c34f-7ea9-42a5-9a04-b4658e8b3ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.554621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d5f5c34f-7ea9-42a5-9a04-b4658e8b3ca1 None None] MFA Rules not processed for user `cf830af8101b4e56877e14e21255f444`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:30.556098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d5f5c34f-7ea9-42a5-9a04-b4658e8b3ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:30.626660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1044/2080] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:30.638258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934a7f8-beef-41e8-ae6c-c2396b40a503 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:30.640131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d934a7f8-beef-41e8-ae6c-c2396b40a503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:30.705791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934a7f8-beef-41e8-ae6c-c2396b40a503 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:30.709663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2d3a4-c90d-4e7b-be49-497895c6f0c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:30.710779 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2d3a4-c90d-4e7b-be49-497895c6f0c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:30.710779 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2d3a4-c90d-4e7b-be49-497895c6f0c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:30.711056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2d3a4-c90d-4e7b-be49-497895c6f0c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:30.720318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2d3a4-c90d-4e7b-be49-497895c6f0c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:30.721799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2d3a4-c90d-4e7b-be49-497895c6f0c1 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:30.732236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1037/2081] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:40.318213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5c34f-7ea9-42a5-9a04-b4658e8b3ca1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:40.322814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:40.390794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5c34f-7ea9-42a5-9a04-b4658e8b3ca1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:40.396572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:40.396801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:40.397077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:40.397350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] PATH_INFO: `/v3/users/cf830af8101b4e56877e14e21255f444` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:40.398731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf830af8101b4e56877e14e21255f444)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:40.400972 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:40.461759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf830af8101b4e56877e14e21255f444'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:40.462910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf830af8101b4e56877e14e21255f444'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:40.465661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf830af8101b4e56877e14e21255f444'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:40.475854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf830af8101b4e56877e14e21255f444'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:40.492366 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf830af8101b4e56877e14e21255f444'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:40.535897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1045/2082] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:40 2022] DELETE /identity/v3/users/cf830af8101b4e56877e14e21255f444 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:40.538111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c496d-252b-4493-9acb-30e13db3a364 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:40.538907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c496d-252b-4493-9acb-30e13db3a364 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:40.539321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c496d-252b-4493-9acb-30e13db3a364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:40.567504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7e7c496d-252b-4493-9acb-30e13db3a364 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:40.569163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7e7c496d-252b-4493-9acb-30e13db3a364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:40.672426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1038/2083] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:40.684942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:40.688412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:40.761148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fca891-4daa-417c-838c-691202cf6f81 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:40.767000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9543951-8c2c-4647-819d-b74ee63092b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:40.767755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9543951-8c2c-4647-819d-b74ee63092b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:40.768095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9543951-8c2c-4647-819d-b74ee63092b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:40.768590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9543951-8c2c-4647-819d-b74ee63092b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:40.781007 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9543951-8c2c-4647-819d-b74ee63092b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:40.782183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9543951-8c2c-4647-819d-b74ee63092b7 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:40.794840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1046/2084] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:41.349781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c496d-252b-4493-9acb-30e13db3a364 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:41.351371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7e7c496d-252b-4493-9acb-30e13db3a364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:41.390320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c496d-252b-4493-9acb-30e13db3a364 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:41.410565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:41.412337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:41.413415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:41.413681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] PATH_INFO: `/v3/projects/7ee621ef88e04e25a9c8aa3e54e3f98e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:41.417698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee621ef88e04e25a9c8aa3e54e3f98e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:41.420004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:41.504453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:41.508503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee621ef88e04e25a9c8aa3e54e3f98e', 'request_id': 'req-b4fbb4b0-7453-4934-adc2-e559364763fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:41.526274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee621ef88e04e25a9c8aa3e54e3f98e', 'request_id': 'req-b4fbb4b0-7453-4934-adc2-e559364763fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:41.544963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee621ef88e04e25a9c8aa3e54e3f98e', 'request_id': 'req-b4fbb4b0-7453-4934-adc2-e559364763fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:41.577170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] The token cache is being invalidate because project 7ee621ef88e04e25a9c8aa3e54e3f98e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:41.577684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:41.578592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4fbb4b0-7453-4934-adc2-e559364763fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:41.581255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1039/2085] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:41 2022] DELETE /identity/v3/projects/7ee621ef88e04e25a9c8aa3e54e3f98e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:41.589646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00acf02-2188-41ac-bf10-2b892885abcd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:41.591355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00acf02-2188-41ac-bf10-2b892885abcd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:41.591642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00acf02-2188-41ac-bf10-2b892885abcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:41.635454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e00acf02-2188-41ac-bf10-2b892885abcd None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:41.638577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e00acf02-2188-41ac-bf10-2b892885abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:41.665239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:41.735726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1047/2086] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:41.745795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487c3f2-8de5-4b8d-86fd-0584d96742fd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:41.746282 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487c3f2-8de5-4b8d-86fd-0584d96742fd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:41.746643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487c3f2-8de5-4b8d-86fd-0584d96742fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:41.768110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e487c3f2-8de5-4b8d-86fd-0584d96742fd None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:41.769389 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e487c3f2-8de5-4b8d-86fd-0584d96742fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:41.780468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1040/2087] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:41.784736 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00acf02-2188-41ac-bf10-2b892885abcd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:41.795893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00acf02-2188-41ac-bf10-2b892885abcd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:41.800861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d86e-8b32-48a2-979c-1859e27844be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:41.801480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d86e-8b32-48a2-979c-1859e27844be admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:41.801764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d86e-8b32-48a2-979c-1859e27844be admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:41.802037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d86e-8b32-48a2-979c-1859e27844be admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:41.802692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698d86e-8b32-48a2-979c-1859e27844be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:24:41.802996 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698d86e-8b32-48a2-979c-1859e27844be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:41.804654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698d86e-8b32-48a2-979c-1859e27844be admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:41.818243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1048/2088] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:24:41.823810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487c3f2-8de5-4b8d-86fd-0584d96742fd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:41.833851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487c3f2-8de5-4b8d-86fd-0584d96742fd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:41.841418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2093b-6afd-45fd-b7bc-895672cca928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:41.842287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2093b-6afd-45fd-b7bc-895672cca928 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:41.842744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2093b-6afd-45fd-b7bc-895672cca928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:41.843013 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2093b-6afd-45fd-b7bc-895672cca928 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:41.843927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2093b-6afd-45fd-b7bc-895672cca928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:41.847065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2093b-6afd-45fd-b7bc-895672cca928 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:41.880192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1041/2089] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:41 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:41.884655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d86e-8b32-48a2-979c-1859e27844be admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:41.899457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d86e-8b32-48a2-979c-1859e27844be admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:41.983148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf78d65-0597-447a-b9c8-3fba30f1ce90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:41.984407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78d65-0597-447a-b9c8-3fba30f1ce90 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:41.984407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78d65-0597-447a-b9c8-3fba30f1ce90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:41.984792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78d65-0597-447a-b9c8-3fba30f1ce90 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:41.985558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf78d65-0597-447a-b9c8-3fba30f1ce90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:41.987613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf78d65-0597-447a-b9c8-3fba30f1ce90 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:42.019845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1049/2090] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:41 2022] POST /identity/v3/users => generated 284 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:42.024425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2093b-6afd-45fd-b7bc-895672cca928 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:42.044537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2093b-6afd-45fd-b7bc-895672cca928 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:42.050701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fd6a0-e7d6-4768-9e1f-ed32efcb654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:42.051613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fd6a0-e7d6-4768-9e1f-ed32efcb654b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:42.052067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fd6a0-e7d6-4768-9e1f-ed32efcb654b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:42.052544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fd6a0-e7d6-4768-9e1f-ed32efcb654b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:42.053569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fd6a0-e7d6-4768-9e1f-ed32efcb654b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:42.055592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fd6a0-e7d6-4768-9e1f-ed32efcb654b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:42.079080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1042/2091] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:42.080092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf78d65-0597-447a-b9c8-3fba30f1ce90 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:42.095284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf78d65-0597-447a-b9c8-3fba30f1ce90 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:42.102005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437ea23-894e-46f4-8adb-6cbeb7f9e1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:42.102554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ea23-894e-46f4-8adb-6cbeb7f9e1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:42.102832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ea23-894e-46f4-8adb-6cbeb7f9e1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:42.103103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ea23-894e-46f4-8adb-6cbeb7f9e1b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:42.103780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437ea23-894e-46f4-8adb-6cbeb7f9e1b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:42.105935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437ea23-894e-46f4-8adb-6cbeb7f9e1b4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:42.120265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1050/2092] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:24:42.125000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fd6a0-e7d6-4768-9e1f-ed32efcb654b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:42.136182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fd6a0-e7d6-4768-9e1f-ed32efcb654b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:42.145921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86742e6-6615-4e8d-abdb-90bdb1133d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:42.146916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86742e6-6615-4e8d-abdb-90bdb1133d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:42.146916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86742e6-6615-4e8d-abdb-90bdb1133d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:42.147121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86742e6-6615-4e8d-abdb-90bdb1133d6e admin admin] PATH_INFO: `/v3/projects/de8ee4e6cc2a4f49befed0b4c474349c/users/eebe100763294230bcf623d54cea890b/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:42.169335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86742e6-6615-4e8d-abdb-90bdb1133d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8ee4e6cc2a4f49befed0b4c474349c, user_id=eebe100763294230bcf623d54cea890b, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:42.173656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86742e6-6615-4e8d-abdb-90bdb1133d6e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:42.200548 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1043/2093] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:42 2022] PUT /identity/v3/projects/de8ee4e6cc2a4f49befed0b4c474349c/users/eebe100763294230bcf623d54cea890b/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:24:42.209801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd342-9fdd-458b-9a11-c71563860741 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:42.210188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd342-9fdd-458b-9a11-c71563860741 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:42.210671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd342-9fdd-458b-9a11-c71563860741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:42.263920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a3abd342-9fdd-458b-9a11-c71563860741 None None] MFA Rules not processed for user `eebe100763294230bcf623d54cea890b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:42.265263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3abd342-9fdd-458b-9a11-c71563860741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:42.370766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1051/2094] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:42.383046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86742e6-6615-4e8d-abdb-90bdb1133d6e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:42.385604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a86742e6-6615-4e8d-abdb-90bdb1133d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:42.481709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86742e6-6615-4e8d-abdb-90bdb1133d6e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:42.486474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7512-e3d8-4319-8a90-f921a0c6eba3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0dfcbad22348fdaea32d64557e68a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:42.487117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7512-e3d8-4319-8a90-f921a0c6eba3 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:42.487400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7512-e3d8-4319-8a90-f921a0c6eba3 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:42.487865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7512-e3d8-4319-8a90-f921a0c6eba3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:42.500439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7512-e3d8-4319-8a90-f921a0c6eba3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:42.502188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7512-e3d8-4319-8a90-f921a0c6eba3 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:42.515057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1044/2095] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:44.780201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd342-9fdd-458b-9a11-c71563860741 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:44.781784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3abd342-9fdd-458b-9a11-c71563860741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:44.804189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:44.850949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd342-9fdd-458b-9a11-c71563860741 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:44.856305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:44.856958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:44.857233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:44.857502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] PATH_INFO: `/v3/users/58fa0eb8e5ad41038d447c54bad91a6d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:44.858614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58fa0eb8e5ad41038d447c54bad91a6d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:44.860589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:44.907677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58fa0eb8e5ad41038d447c54bad91a6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:44.908333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58fa0eb8e5ad41038d447c54bad91a6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:44.908967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58fa0eb8e5ad41038d447c54bad91a6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:44.915928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58fa0eb8e5ad41038d447c54bad91a6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:44.929083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2379059d-428d-4ae9-b930-8845b78b2485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58fa0eb8e5ad41038d447c54bad91a6d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:44.945680 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1052/2096] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:44 2022] DELETE /identity/v3/users/58fa0eb8e5ad41038d447c54bad91a6d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:45.128657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7512-e3d8-4319-8a90-f921a0c6eba3 service glance] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:45.141560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7512-e3d8-4319-8a90-f921a0c6eba3 service glance] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:45.215577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:45.216240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.216534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.216762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] PATH_INFO: `/v3/projects/ea214b3e591a44928fd7af5f51095405` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.217894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea214b3e591a44928fd7af5f51095405)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:45.219841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:45.239953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:45.273362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea214b3e591a44928fd7af5f51095405', 'request_id': 'req-5ad1c418-33d7-4925-94c0-cb528c306b26'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:45.279488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea214b3e591a44928fd7af5f51095405', 'request_id': 'req-5ad1c418-33d7-4925-94c0-cb528c306b26'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:45.287890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea214b3e591a44928fd7af5f51095405', 'request_id': 'req-5ad1c418-33d7-4925-94c0-cb528c306b26'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:45.300332 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] The token cache is being invalidate because project ea214b3e591a44928fd7af5f51095405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:45.300764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:45.301536 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ad1c418-33d7-4925-94c0-cb528c306b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:45.303268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1045/2097] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:45 2022] DELETE /identity/v3/projects/ea214b3e591a44928fd7af5f51095405 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:45.311498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbeb8ef-9e6e-4d50-b22b-ff33e1fe7583 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.312004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbeb8ef-9e6e-4d50-b22b-ff33e1fe7583 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.312550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbeb8ef-9e6e-4d50-b22b-ff33e1fe7583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.341643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bfbeb8ef-9e6e-4d50-b22b-ff33e1fe7583 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:45.342861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bfbeb8ef-9e6e-4d50-b22b-ff33e1fe7583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:45.402864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1053/2098] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:45.410638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c691-01e0-4900-a302-85522c157676 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.410927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c691-01e0-4900-a302-85522c157676 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.411205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c691-01e0-4900-a302-85522c157676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.431454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e562c691-01e0-4900-a302-85522c157676 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:45.434006 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e562c691-01e0-4900-a302-85522c157676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:45.446471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1046/2099] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:45.449975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbeb8ef-9e6e-4d50-b22b-ff33e1fe7583 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:45.461995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbeb8ef-9e6e-4d50-b22b-ff33e1fe7583 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:45.475192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02914a1c-834d-4cf8-81d6-b94667aabd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:45.475874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02914a1c-834d-4cf8-81d6-b94667aabd5e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.476154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02914a1c-834d-4cf8-81d6-b94667aabd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.476600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02914a1c-834d-4cf8-81d6-b94667aabd5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.477110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02914a1c-834d-4cf8-81d6-b94667aabd5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:24:45.477407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02914a1c-834d-4cf8-81d6-b94667aabd5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:45.479030 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02914a1c-834d-4cf8-81d6-b94667aabd5e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:45.493525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1054/2100] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:24:45.499039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562c691-01e0-4900-a302-85522c157676 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:45.508869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562c691-01e0-4900-a302-85522c157676 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:45.514097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c8f48-bb5c-4f55-997e-597b58c9668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:45.514699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c8f48-bb5c-4f55-997e-597b58c9668b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.515543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c8f48-bb5c-4f55-997e-597b58c9668b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.515543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c8f48-bb5c-4f55-997e-597b58c9668b admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.515864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c8f48-bb5c-4f55-997e-597b58c9668b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:45.517867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c8f48-bb5c-4f55-997e-597b58c9668b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:45.552749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1047/2101] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:45 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:45.555204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02914a1c-834d-4cf8-81d6-b94667aabd5e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:45.566770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02914a1c-834d-4cf8-81d6-b94667aabd5e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:45.593358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:45.630229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b62fc-a3bd-4508-a3c1-8325e8dab5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:45.630834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b62fc-a3bd-4508-a3c1-8325e8dab5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.631111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b62fc-a3bd-4508-a3c1-8325e8dab5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.631380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b62fc-a3bd-4508-a3c1-8325e8dab5e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.632014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b62fc-a3bd-4508-a3c1-8325e8dab5e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:45.634072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b62fc-a3bd-4508-a3c1-8325e8dab5e5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:45.660199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1055/2102] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:45 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:45.665604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c8f48-bb5c-4f55-997e-597b58c9668b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:45.679936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c8f48-bb5c-4f55-997e-597b58c9668b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:45.684786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84d0fa-6e34-49b3-aaaa-9a193ed4e3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:45.685388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84d0fa-6e34-49b3-aaaa-9a193ed4e3aa admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.685663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84d0fa-6e34-49b3-aaaa-9a193ed4e3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.685930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84d0fa-6e34-49b3-aaaa-9a193ed4e3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.686575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84d0fa-6e34-49b3-aaaa-9a193ed4e3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:45.688209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84d0fa-6e34-49b3-aaaa-9a193ed4e3aa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:45.699570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1048/2103] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:45.709266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b62fc-a3bd-4508-a3c1-8325e8dab5e5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:45.720710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b62fc-a3bd-4508-a3c1-8325e8dab5e5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:45.725328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd027b92-ee73-456a-845e-243cbebacbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:45.725955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd027b92-ee73-456a-845e-243cbebacbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.726228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd027b92-ee73-456a-845e-243cbebacbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.726494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd027b92-ee73-456a-845e-243cbebacbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.727131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd027b92-ee73-456a-845e-243cbebacbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:45.728785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd027b92-ee73-456a-845e-243cbebacbf7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:45.740768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1056/2104] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:45.746297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84d0fa-6e34-49b3-aaaa-9a193ed4e3aa admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:45.763839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84d0fa-6e34-49b3-aaaa-9a193ed4e3aa admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:45.777348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3d9ec-0600-4772-8d15-784c65ec0c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:45.778038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3d9ec-0600-4772-8d15-784c65ec0c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.778369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3d9ec-0600-4772-8d15-784c65ec0c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.778610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3d9ec-0600-4772-8d15-784c65ec0c21 admin admin] PATH_INFO: `/v3/projects/79469e5c56d34cd8bc100c03f4bca745/users/98ca64a3f0084d068555aaf30d2a2a3e/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.797989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3d9ec-0600-4772-8d15-784c65ec0c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79469e5c56d34cd8bc100c03f4bca745, user_id=98ca64a3f0084d068555aaf30d2a2a3e, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:45.800739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3d9ec-0600-4772-8d15-784c65ec0c21 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:45.814712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1049/2105] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:45 2022] PUT /identity/v3/projects/79469e5c56d34cd8bc100c03f4bca745/users/98ca64a3f0084d068555aaf30d2a2a3e/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:45.825384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c334d9d-0b21-4027-b5a2-16764d0836c6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:45.825910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c334d9d-0b21-4027-b5a2-16764d0836c6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:45.825910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c334d9d-0b21-4027-b5a2-16764d0836c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:45.856717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0c334d9d-0b21-4027-b5a2-16764d0836c6 None None] MFA Rules not processed for user `98ca64a3f0084d068555aaf30d2a2a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:45.857950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c334d9d-0b21-4027-b5a2-16764d0836c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:45.928657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1057/2106] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:45.940323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3d9ec-0600-4772-8d15-784c65ec0c21 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:45.942522 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-48b3d9ec-0600-4772-8d15-784c65ec0c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:46.032556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3d9ec-0600-4772-8d15-784c65ec0c21 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:46.038517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52843fb3-5812-434e-8e09-4ee1d05170b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:46.039151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52843fb3-5812-434e-8e09-4ee1d05170b8 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:46.039442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52843fb3-5812-434e-8e09-4ee1d05170b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:46.039717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52843fb3-5812-434e-8e09-4ee1d05170b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:46.049589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52843fb3-5812-434e-8e09-4ee1d05170b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:46.051374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52843fb3-5812-434e-8e09-4ee1d05170b8 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:46.061971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1050/2107] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:52.375766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146c0ee-1650-48cf-98d2-00ea1e2b0019 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:52.376324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146c0ee-1650-48cf-98d2-00ea1e2b0019 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:52.376324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146c0ee-1650-48cf-98d2-00ea1e2b0019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:52.397935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b146c0ee-1650-48cf-98d2-00ea1e2b0019 None None] MFA Rules not processed for user `887eceef50b84dde975e15ac9e7da02b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:52.399312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b146c0ee-1650-48cf-98d2-00ea1e2b0019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:52.486858 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1058/2108] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:54.656823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146c0ee-1650-48cf-98d2-00ea1e2b0019 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:54.657658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52843fb3-5812-434e-8e09-4ee1d05170b8 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:54.658496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b146c0ee-1650-48cf-98d2-00ea1e2b0019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:54.705374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:54.723881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:54.761262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52843fb3-5812-434e-8e09-4ee1d05170b8 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:54.763045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146c0ee-1650-48cf-98d2-00ea1e2b0019 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:54.766696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:54.767592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:54.768132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:54.768718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] PATH_INFO: `/v3/users/98ca64a3f0084d068555aaf30d2a2a3e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:54.769476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:54.770034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ca64a3f0084d068555aaf30d2a2a3e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:54.770149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:54.770392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:54.770673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] PATH_INFO: `/v3/users/764459d91f474c8bac6f108700cf220f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:54.771924 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=764459d91f474c8bac6f108700cf220f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:54.776001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:54.776195 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:54.844817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ca64a3f0084d068555aaf30d2a2a3e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.860548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764459d91f474c8bac6f108700cf220f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.861349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764459d91f474c8bac6f108700cf220f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.862150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ca64a3f0084d068555aaf30d2a2a3e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.862813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764459d91f474c8bac6f108700cf220f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.865464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ca64a3f0084d068555aaf30d2a2a3e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.880717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764459d91f474c8bac6f108700cf220f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.893426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ca64a3f0084d068555aaf30d2a2a3e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.899472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9b6467ff-1757-4203-bb98-d009f4a51cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764459d91f474c8bac6f108700cf220f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.910154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-13272cb0-ff79-4f37-bcf3-672da23de393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ca64a3f0084d068555aaf30d2a2a3e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:54.920222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1051/2109] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:54 2022] DELETE /identity/v3/users/98ca64a3f0084d068555aaf30d2a2a3e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:54.926930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeaece-2445-4b87-9041-8218a2dbb4fa None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:54.928827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeaece-2445-4b87-9041-8218a2dbb4fa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:54.929181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeaece-2445-4b87-9041-8218a2dbb4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:54.942571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1059/2110] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:24:54 2022] DELETE /identity/v3/users/764459d91f474c8bac6f108700cf220f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:54.950206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce502dc-afb0-4220-b0d4-f12119d637cb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:54.950727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce502dc-afb0-4220-b0d4-f12119d637cb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:54.951133 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce502dc-afb0-4220-b0d4-f12119d637cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:54.973211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9adeaece-2445-4b87-9041-8218a2dbb4fa None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:54.974593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9adeaece-2445-4b87-9041-8218a2dbb4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:55.009515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5ce502dc-afb0-4220-b0d4-f12119d637cb None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:55.011783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ce502dc-afb0-4220-b0d4-f12119d637cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:55.084680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1052/2111] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:55.095115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeaece-2445-4b87-9041-8218a2dbb4fa None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:55.097460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9adeaece-2445-4b87-9041-8218a2dbb4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:55.109692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1060/2112] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:55.116872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce502dc-afb0-4220-b0d4-f12119d637cb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:55.121643 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ce502dc-afb0-4220-b0d4-f12119d637cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:55.185596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:55.195492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeaece-2445-4b87-9041-8218a2dbb4fa None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:55.198100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce502dc-afb0-4220-b0d4-f12119d637cb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:55.202538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244710c1-5cd0-4b24-a99e-770307caa1c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:55.203149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244710c1-5cd0-4b24-a99e-770307caa1c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:55.203920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cb564-2e45-4881-9baa-7a1972eced02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:55.204559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cb564-2e45-4881-9baa-7a1972eced02 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:55.204748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244710c1-5cd0-4b24-a99e-770307caa1c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:55.204916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cb564-2e45-4881-9baa-7a1972eced02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:55.205057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244710c1-5cd0-4b24-a99e-770307caa1c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:55.205196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cb564-2e45-4881-9baa-7a1972eced02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:55.216626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cb564-2e45-4881-9baa-7a1972eced02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:55.216953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244710c1-5cd0-4b24-a99e-770307caa1c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:55.218007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cb564-2e45-4881-9baa-7a1972eced02 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:55.218417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244710c1-5cd0-4b24-a99e-770307caa1c2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:55.232031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1053/2113] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:24:55.239818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1061/2114] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:24:55.789809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cb564-2e45-4881-9baa-7a1972eced02 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:55.793369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-617cb564-2e45-4881-9baa-7a1972eced02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:55.811967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cb564-2e45-4881-9baa-7a1972eced02 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:55.818726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:55.819317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:55.819602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:55.819881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] PATH_INFO: `/v3/projects/90628c1360644da79e6825261c5ae2fb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:55.821250 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90628c1360644da79e6825261c5ae2fb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:55.823217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:55.914047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90628c1360644da79e6825261c5ae2fb', 'request_id': 'req-f847bd58-a323-480e-95cd-4889b0ec9e66'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:55.924892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90628c1360644da79e6825261c5ae2fb', 'request_id': 'req-f847bd58-a323-480e-95cd-4889b0ec9e66'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:55.931424 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90628c1360644da79e6825261c5ae2fb', 'request_id': 'req-f847bd58-a323-480e-95cd-4889b0ec9e66'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:55.937719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244710c1-5cd0-4b24-a99e-770307caa1c2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:55.939317 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-244710c1-5cd0-4b24-a99e-770307caa1c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:55.952745 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:55.960630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244710c1-5cd0-4b24-a99e-770307caa1c2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:55.960771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] The token cache is being invalidate because project 90628c1360644da79e6825261c5ae2fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:55.961214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:55.962589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f847bd58-a323-480e-95cd-4889b0ec9e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:55.967070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1054/2115] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:55 2022] DELETE /identity/v3/projects/90628c1360644da79e6825261c5ae2fb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:55.974623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944e03e-b9e2-47e3-a9dc-172c9d72ac9b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:55.974811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944e03e-b9e2-47e3-a9dc-172c9d72ac9b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:55.975211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944e03e-b9e2-47e3-a9dc-172c9d72ac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.010396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0944e03e-b9e2-47e3-a9dc-172c9d72ac9b None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:56.012377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0944e03e-b9e2-47e3-a9dc-172c9d72ac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:56.084299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.085097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.085484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.085703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] PATH_INFO: `/v3/projects/79469e5c56d34cd8bc100c03f4bca745` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.093799 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79469e5c56d34cd8bc100c03f4bca745)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.099341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.132428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1055/2116] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:24:56.132428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebd4f5-a832-4f64-bafc-7b8cea667847 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.132428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebd4f5-a832-4f64-bafc-7b8cea667847 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.132428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebd4f5-a832-4f64-bafc-7b8cea667847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.154192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f9ebd4f5-a832-4f64-bafc-7b8cea667847 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:56.156265 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f9ebd4f5-a832-4f64-bafc-7b8cea667847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:56.165273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1056/2117] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:56.170382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebd4f5-a832-4f64-bafc-7b8cea667847 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.184896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebd4f5-a832-4f64-bafc-7b8cea667847 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.217343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79469e5c56d34cd8bc100c03f4bca745', 'request_id': 'req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:56.226644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79469e5c56d34cd8bc100c03f4bca745', 'request_id': 'req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:56.240078 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79469e5c56d34cd8bc100c03f4bca745', 'request_id': 'req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:56.258843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] The token cache is being invalidate because project 79469e5c56d34cd8bc100c03f4bca745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:24:56.259046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:56.259898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-eb61ab67-a41f-4b9e-a767-fa0e4e884f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:24:56.261897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1062/2118] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:24:55 2022] DELETE /identity/v3/projects/79469e5c56d34cd8bc100c03f4bca745 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:24:56.268271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5004f-3f58-4ee5-acd1-e1bb555e389f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.269308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5004f-3f58-4ee5-acd1-e1bb555e389f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.269688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5004f-3f58-4ee5-acd1-e1bb555e389f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.285744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c587b0-8277-4322-a20e-0fec46645dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.287677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c587b0-8277-4322-a20e-0fec46645dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.288149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c587b0-8277-4322-a20e-0fec46645dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.288606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c587b0-8277-4322-a20e-0fec46645dd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.289808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c587b0-8277-4322-a20e-0fec46645dd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:24:56.290114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c587b0-8277-4322-a20e-0fec46645dd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.292997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c587b0-8277-4322-a20e-0fec46645dd6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.300955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6ae5004f-3f58-4ee5-acd1-e1bb555e389f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:56.302520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6ae5004f-3f58-4ee5-acd1-e1bb555e389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:56.304205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:56.312729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1063/2119] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:56.314067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5004f-3f58-4ee5-acd1-e1bb555e389f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.315672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6ae5004f-3f58-4ee5-acd1-e1bb555e389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:56.320163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1057/2120] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:24:56.322174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff36cee-3a28-4f85-be18-af562e027df4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.322446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff36cee-3a28-4f85-be18-af562e027df4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.322728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff36cee-3a28-4f85-be18-af562e027df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.330487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5004f-3f58-4ee5-acd1-e1bb555e389f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.338358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c0a31-3f99-4bda-8b18-4868f54bc474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.339095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c0a31-3f99-4bda-8b18-4868f54bc474 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.339978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c0a31-3f99-4bda-8b18-4868f54bc474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.340314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c0a31-3f99-4bda-8b18-4868f54bc474 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.341082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c0a31-3f99-4bda-8b18-4868f54bc474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.343281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c0a31-3f99-4bda-8b18-4868f54bc474 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.346949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-dff36cee-3a28-4f85-be18-af562e027df4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:56.348185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dff36cee-3a28-4f85-be18-af562e027df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:56.361066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1058/2121] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:56.362130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff36cee-3a28-4f85-be18-af562e027df4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.374297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff36cee-3a28-4f85-be18-af562e027df4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.381381 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1064/2122] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:56 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:56.385537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c0a31-3f99-4bda-8b18-4868f54bc474 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.387092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cf298-0b79-41e2-b763-53e85cbc149d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.388217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cf298-0b79-41e2-b763-53e85cbc149d admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.388614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cf298-0b79-41e2-b763-53e85cbc149d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.388870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cf298-0b79-41e2-b763-53e85cbc149d admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.389539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cf298-0b79-41e2-b763-53e85cbc149d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:24:56.390275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cf298-0b79-41e2-b763-53e85cbc149d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.392566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cf298-0b79-41e2-b763-53e85cbc149d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.411901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1059/2123] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:24:56.416264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cf298-0b79-41e2-b763-53e85cbc149d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.428180 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cf298-0b79-41e2-b763-53e85cbc149d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.468268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c0a31-3f99-4bda-8b18-4868f54bc474 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.473203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a899d-a9e1-499b-880f-9bc498750f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.473821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a899d-a9e1-499b-880f-9bc498750f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.474121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a899d-a9e1-499b-880f-9bc498750f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.474383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a899d-a9e1-499b-880f-9bc498750f14 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.475023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a899d-a9e1-499b-880f-9bc498750f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.477124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a899d-a9e1-499b-880f-9bc498750f14 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.488859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:56.493348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca61153-1831-44d8-9df4-26263e8545a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.494011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca61153-1831-44d8-9df4-26263e8545a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.494324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca61153-1831-44d8-9df4-26263e8545a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.494583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca61153-1831-44d8-9df4-26263e8545a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.495274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca61153-1831-44d8-9df4-26263e8545a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.498082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca61153-1831-44d8-9df4-26263e8545a2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.507769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1065/2124] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:56 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:56.508742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a899d-a9e1-499b-880f-9bc498750f14 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.520186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a899d-a9e1-499b-880f-9bc498750f14 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.525186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693d8d9-d0b9-4f22-827a-515609efdcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.525803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693d8d9-d0b9-4f22-827a-515609efdcab admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.526085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693d8d9-d0b9-4f22-827a-515609efdcab admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.526358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693d8d9-d0b9-4f22-827a-515609efdcab admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.527090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693d8d9-d0b9-4f22-827a-515609efdcab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.531127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1060/2125] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:56 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:56.531270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693d8d9-d0b9-4f22-827a-515609efdcab admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.532370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca61153-1831-44d8-9df4-26263e8545a2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.539637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1066/2126] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:56.543854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693d8d9-d0b9-4f22-827a-515609efdcab admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.553943 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693d8d9-d0b9-4f22-827a-515609efdcab admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.607739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca61153-1831-44d8-9df4-26263e8545a2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.612744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa30553-d7b8-425e-af7a-be5c5c604ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.613356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa30553-d7b8-425e-af7a-be5c5c604ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.614181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa30553-d7b8-425e-af7a-be5c5c604ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.614181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa30553-d7b8-425e-af7a-be5c5c604ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.614527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa30553-d7b8-425e-af7a-be5c5c604ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.616604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa30553-d7b8-425e-af7a-be5c5c604ab2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.617155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65ca05-e4dc-494f-8f2f-ff6784b63fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.617747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65ca05-e4dc-494f-8f2f-ff6784b63fbf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.618025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65ca05-e4dc-494f-8f2f-ff6784b63fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.618935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65ca05-e4dc-494f-8f2f-ff6784b63fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.618935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65ca05-e4dc-494f-8f2f-ff6784b63fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.622053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65ca05-e4dc-494f-8f2f-ff6784b63fbf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.641056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1067/2127] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:24:56.647993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65ca05-e4dc-494f-8f2f-ff6784b63fbf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.648468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1061/2128] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:56 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:56.649733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa30553-d7b8-425e-af7a-be5c5c604ab2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.661932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65ca05-e4dc-494f-8f2f-ff6784b63fbf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.666751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa30553-d7b8-425e-af7a-be5c5c604ab2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.673650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1734fca-29ba-454f-9b57-261651d715fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.673650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734fca-29ba-454f-9b57-261651d715fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.673650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734fca-29ba-454f-9b57-261651d715fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.673650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1734fca-29ba-454f-9b57-261651d715fd admin admin] PATH_INFO: `/v3/projects/42b848bb5725477b8832057b56c9cf4d/users/1be80a6a93454cd8b7128e976308bd4b/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.686274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8259f-f177-4742-beb1-b94e836331b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.686958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8259f-f177-4742-beb1-b94e836331b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.687248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8259f-f177-4742-beb1-b94e836331b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.687530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8259f-f177-4742-beb1-b94e836331b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.688207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8259f-f177-4742-beb1-b94e836331b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.689921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8259f-f177-4742-beb1-b94e836331b1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.695537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1734fca-29ba-454f-9b57-261651d715fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b848bb5725477b8832057b56c9cf4d, user_id=1be80a6a93454cd8b7128e976308bd4b, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.698803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1734fca-29ba-454f-9b57-261651d715fd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.703715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1062/2129] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:56.704869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8259f-f177-4742-beb1-b94e836331b1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.721875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8259f-f177-4742-beb1-b94e836331b1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.727033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1068/2130] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:56 2022] PUT /identity/v3/projects/42b848bb5725477b8832057b56c9cf4d/users/1be80a6a93454cd8b7128e976308bd4b/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:24:56.734128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba48b91-5e24-4deb-b8ac-90f59c2c587e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.734657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba48b91-5e24-4deb-b8ac-90f59c2c587e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.735319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba48b91-5e24-4deb-b8ac-90f59c2c587e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.762833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3ba48b91-5e24-4deb-b8ac-90f59c2c587e None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:56.764046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3ba48b91-5e24-4deb-b8ac-90f59c2c587e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:56.788873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:56.803802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9901e-04ba-421d-a93a-5114c6c0102f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.804565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9901e-04ba-421d-a93a-5114c6c0102f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.804821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9901e-04ba-421d-a93a-5114c6c0102f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.805100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9901e-04ba-421d-a93a-5114c6c0102f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.805842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9901e-04ba-421d-a93a-5114c6c0102f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.807656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9901e-04ba-421d-a93a-5114c6c0102f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.822891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1063/2131] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:56.824610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9901e-04ba-421d-a93a-5114c6c0102f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.834354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9901e-04ba-421d-a93a-5114c6c0102f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.835659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1069/2132] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:56.847918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a47f39-533a-4be3-ba45-58337230fcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.847918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a47f39-533a-4be3-ba45-58337230fcaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.847918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a47f39-533a-4be3-ba45-58337230fcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.847918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a47f39-533a-4be3-ba45-58337230fcaa admin admin] PATH_INFO: `/v3/projects/1189d7a65d4346b7ab7b0302bbb5ec43/users/a96a9139c69d460684958eff3f40d974/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.848407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba48b91-5e24-4deb-b8ac-90f59c2c587e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:56.848407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3ba48b91-5e24-4deb-b8ac-90f59c2c587e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:56.857830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a47f39-533a-4be3-ba45-58337230fcaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1189d7a65d4346b7ab7b0302bbb5ec43, user_id=a96a9139c69d460684958eff3f40d974, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:56.860555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a47f39-533a-4be3-ba45-58337230fcaa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:56.882304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1064/2133] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:56 2022] PUT /identity/v3/projects/1189d7a65d4346b7ab7b0302bbb5ec43/users/a96a9139c69d460684958eff3f40d974/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:56.894793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b616f-dd68-4e2b-8b2b-74d69d5c947d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.894793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b616f-dd68-4e2b-8b2b-74d69d5c947d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.894793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b616f-dd68-4e2b-8b2b-74d69d5c947d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:56.914250 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba48b91-5e24-4deb-b8ac-90f59c2c587e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:56.923606 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:56.928600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-db4b616f-dd68-4e2b-8b2b-74d69d5c947d None None] MFA Rules not processed for user `a96a9139c69d460684958eff3f40d974`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:56.929979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-db4b616f-dd68-4e2b-8b2b-74d69d5c947d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:56.972518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccd39b-0b35-40cd-a562-7c35d28beb3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:56.973129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccd39b-0b35-40cd-a562-7c35d28beb3c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:56.973406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccd39b-0b35-40cd-a562-7c35d28beb3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:56.973682 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccd39b-0b35-40cd-a562-7c35d28beb3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.002062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1065/2134] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:57.006847 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b616f-dd68-4e2b-8b2b-74d69d5c947d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:57.048667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ccd39b-0b35-40cd-a562-7c35d28beb3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:57.050339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ccd39b-0b35-40cd-a562-7c35d28beb3c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:57.062874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1070/2135] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:24:57.075296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b616f-dd68-4e2b-8b2b-74d69d5c947d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:57.080335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b600a1-a382-4d94-9693-d9d0413685fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:57.081071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b600a1-a382-4d94-9693-d9d0413685fd admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:57.081419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b600a1-a382-4d94-9693-d9d0413685fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:57.081759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b600a1-a382-4d94-9693-d9d0413685fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.082403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b600a1-a382-4d94-9693-d9d0413685fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:57.084413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b600a1-a382-4d94-9693-d9d0413685fd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:57.121685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1066/2136] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:24:57 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:24:57.122668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccd39b-0b35-40cd-a562-7c35d28beb3c service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:57.136704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccd39b-0b35-40cd-a562-7c35d28beb3c service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:57.227902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826d9e6-7438-49d1-bd70-d2987f2b77a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:57.228565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826d9e6-7438-49d1-bd70-d2987f2b77a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:57.228848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826d9e6-7438-49d1-bd70-d2987f2b77a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:57.229124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826d9e6-7438-49d1-bd70-d2987f2b77a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.230471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826d9e6-7438-49d1-bd70-d2987f2b77a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:57.232786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826d9e6-7438-49d1-bd70-d2987f2b77a8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:57.256602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1071/2137] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:24:57 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:24:57.260361 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b600a1-a382-4d94-9693-d9d0413685fd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:57.275182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b600a1-a382-4d94-9693-d9d0413685fd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:57.281000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeda69-f830-4242-adfe-8281a61b442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:57.281735 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeda69-f830-4242-adfe-8281a61b442b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:57.282075 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeda69-f830-4242-adfe-8281a61b442b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:57.282406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeda69-f830-4242-adfe-8281a61b442b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.283150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeda69-f830-4242-adfe-8281a61b442b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:57.284918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeda69-f830-4242-adfe-8281a61b442b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:57.302274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1067/2138] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:24:57.303352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826d9e6-7438-49d1-bd70-d2987f2b77a8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:57.315020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826d9e6-7438-49d1-bd70-d2987f2b77a8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:57.322635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719c146-b30c-46dc-ac42-0acef969b17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:57.323305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c146-b30c-46dc-ac42-0acef969b17f admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:57.323660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c146-b30c-46dc-ac42-0acef969b17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:57.324656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c146-b30c-46dc-ac42-0acef969b17f admin admin] PATH_INFO: `/v3/projects/b0ac82d697db48c38e4cd023f5242668/users/2c058c15ca3a432eaee840384ae0548b/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.349794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719c146-b30c-46dc-ac42-0acef969b17f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ac82d697db48c38e4cd023f5242668, user_id=2c058c15ca3a432eaee840384ae0548b, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:57.352933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719c146-b30c-46dc-ac42-0acef969b17f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:57.370549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1072/2139] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:57 2022] PUT /identity/v3/projects/b0ac82d697db48c38e4cd023f5242668/users/2c058c15ca3a432eaee840384ae0548b/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:57.374352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeda69-f830-4242-adfe-8281a61b442b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:57.385299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeda69-f830-4242-adfe-8281a61b442b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:57.406329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:57.454519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda7e50-d5c2-4e71-9f91-bc8bc93af095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:57.455147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda7e50-d5c2-4e71-9f91-bc8bc93af095 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:57.455432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda7e50-d5c2-4e71-9f91-bc8bc93af095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:57.455859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda7e50-d5c2-4e71-9f91-bc8bc93af095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.456597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda7e50-d5c2-4e71-9f91-bc8bc93af095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:57.458699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda7e50-d5c2-4e71-9f91-bc8bc93af095 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:57.471503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1068/2140] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:24:57.475492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719c146-b30c-46dc-ac42-0acef969b17f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:57.490146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719c146-b30c-46dc-ac42-0acef969b17f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:57.495064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5525daf-877e-4c60-a06e-ac2a05f45240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:57.495975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5525daf-877e-4c60-a06e-ac2a05f45240 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:57.496254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5525daf-877e-4c60-a06e-ac2a05f45240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:57.496560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5525daf-877e-4c60-a06e-ac2a05f45240 admin admin] PATH_INFO: `/v3/projects/b0ac82d697db48c38e4cd023f5242668/users/2c058c15ca3a432eaee840384ae0548b/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.505339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5525daf-877e-4c60-a06e-ac2a05f45240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ac82d697db48c38e4cd023f5242668, user_id=2c058c15ca3a432eaee840384ae0548b, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:57.508284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5525daf-877e-4c60-a06e-ac2a05f45240 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:57.522808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1073/2141] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:24:57 2022] PUT /identity/v3/projects/b0ac82d697db48c38e4cd023f5242668/users/2c058c15ca3a432eaee840384ae0548b/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:24:57.533507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270185a-292e-49f9-a223-0170010dbb19 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:57.533860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270185a-292e-49f9-a223-0170010dbb19 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:57.534076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270185a-292e-49f9-a223-0170010dbb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.584350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1270185a-292e-49f9-a223-0170010dbb19 None None] MFA Rules not processed for user `2c058c15ca3a432eaee840384ae0548b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:57.585924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1270185a-292e-49f9-a223-0170010dbb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:57.673690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1069/2142] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:57.682740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5525daf-877e-4c60-a06e-ac2a05f45240 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:57.685172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d5525daf-877e-4c60-a06e-ac2a05f45240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:57.701534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:24:57.749316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5525daf-877e-4c60-a06e-ac2a05f45240 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:57.754278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b09253-ab54-47a2-b14f-9f2de3b91a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:57.754916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09253-ab54-47a2-b14f-9f2de3b91a69 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:57.755198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09253-ab54-47a2-b14f-9f2de3b91a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:57.755478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09253-ab54-47a2-b14f-9f2de3b91a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.765865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b09253-ab54-47a2-b14f-9f2de3b91a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:57.767331 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b09253-ab54-47a2-b14f-9f2de3b91a69 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:57.777863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1074/2143] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:57.873085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270185a-292e-49f9-a223-0170010dbb19 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:57.891103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270185a-292e-49f9-a223-0170010dbb19 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:57.895539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af9250-2196-40a4-9de7-f9478d628b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:57.896164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af9250-2196-40a4-9de7-f9478d628b04 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:57.896464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af9250-2196-40a4-9de7-f9478d628b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:57.896743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af9250-2196-40a4-9de7-f9478d628b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:57.973878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af9250-2196-40a4-9de7-f9478d628b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:57.975563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af9250-2196-40a4-9de7-f9478d628b04 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:57.986304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1070/2144] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:58.318426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbd429-a96f-4b63-93fb-23d76d335dec None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:58.318953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbd429-a96f-4b63-93fb-23d76d335dec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:58.319072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbd429-a96f-4b63-93fb-23d76d335dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:58.351528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-84bbd429-a96f-4b63-93fb-23d76d335dec None None] MFA Rules not processed for user `1be80a6a93454cd8b7128e976308bd4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:58.355623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-84bbd429-a96f-4b63-93fb-23d76d335dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:58.435860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1075/2145] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:24:58.446836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af9250-2196-40a4-9de7-f9478d628b04 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:24:58.448238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e5af9250-2196-40a4-9de7-f9478d628b04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:58.462376 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af9250-2196-40a4-9de7-f9478d628b04 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:24:58.467188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74017c-1889-47b4-bb19-1aaf4048df99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:24:58.467796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74017c-1889-47b4-bb19-1aaf4048df99 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:58.468085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74017c-1889-47b4-bb19-1aaf4048df99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:58.468361 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74017c-1889-47b4-bb19-1aaf4048df99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:58.479114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74017c-1889-47b4-bb19-1aaf4048df99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:24:58.481948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74017c-1889-47b4-bb19-1aaf4048df99 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:24:58.491671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1071/2146] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:24:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:24:59.890579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79f25e-9aa3-44ac-ab66-c313a9682688 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:24:59.891143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79f25e-9aa3-44ac-ab66-c313a9682688 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:24:59.891143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79f25e-9aa3-44ac-ab66-c313a9682688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:24:59.911877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1e79f25e-9aa3-44ac-ab66-c313a9682688 None None] MFA Rules not processed for user `1be80a6a93454cd8b7128e976308bd4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:24:59.913137 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1e79f25e-9aa3-44ac-ab66-c313a9682688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:24:59.922338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1076/2147] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:24:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:08.355299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74017c-1889-47b4-bb19-1aaf4048df99 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:08.397194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:08.457604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74017c-1889-47b4-bb19-1aaf4048df99 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:08.465483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:08.466347 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:08.469105 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:08.469718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] PATH_INFO: `/v3/users/a96a9139c69d460684958eff3f40d974` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:08.472412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96a9139c69d460684958eff3f40d974)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:08.475065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:08.548479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96a9139c69d460684958eff3f40d974'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:08.559330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96a9139c69d460684958eff3f40d974'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:08.560324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96a9139c69d460684958eff3f40d974'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:08.592202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96a9139c69d460684958eff3f40d974'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:08.600449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96a9139c69d460684958eff3f40d974'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:08.607833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1072/2148] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:08 2022] DELETE /identity/v3/users/a96a9139c69d460684958eff3f40d974 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:08.612514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff2ecb-2168-43bc-94e2-346680e63782 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:08.612995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff2ecb-2168-43bc-94e2-346680e63782 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:08.613087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff2ecb-2168-43bc-94e2-346680e63782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:08.649271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1aff2ecb-2168-43bc-94e2-346680e63782 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:08.650450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1aff2ecb-2168-43bc-94e2-346680e63782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:08.717616 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1077/2149] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:25:08.727834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:08.736720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:08.796134 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15597f2-2b31-4970-a889-55533ec40fae admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:08.801222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d90a0-91a5-46a3-96fa-c1fb0c4ef0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:08.801820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d90a0-91a5-46a3-96fa-c1fb0c4ef0aa service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:08.802135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d90a0-91a5-46a3-96fa-c1fb0c4ef0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:08.802425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d90a0-91a5-46a3-96fa-c1fb0c4ef0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:08.813449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d90a0-91a5-46a3-96fa-c1fb0c4ef0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:08.814896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d90a0-91a5-46a3-96fa-c1fb0c4ef0aa service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:08.827203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1073/2150] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:09.134899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff2ecb-2168-43bc-94e2-346680e63782 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:09.136505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1aff2ecb-2168-43bc-94e2-346680e63782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:09.149540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff2ecb-2168-43bc-94e2-346680e63782 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:09.155304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:09.155947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:09.156225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:09.156542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] PATH_INFO: `/v3/projects/1189d7a65d4346b7ab7b0302bbb5ec43` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:09.157677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1189d7a65d4346b7ab7b0302bbb5ec43)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:09.160292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:09.210000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:09.225156 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1189d7a65d4346b7ab7b0302bbb5ec43', 'request_id': 'req-0394bdc1-a939-457b-8a26-d3218ae2c1fa'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.233706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1189d7a65d4346b7ab7b0302bbb5ec43', 'request_id': 'req-0394bdc1-a939-457b-8a26-d3218ae2c1fa'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.246687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1189d7a65d4346b7ab7b0302bbb5ec43', 'request_id': 'req-0394bdc1-a939-457b-8a26-d3218ae2c1fa'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.255408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] The token cache is being invalidate because project 1189d7a65d4346b7ab7b0302bbb5ec43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:09.255696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.256383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.257939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1078/2151] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:09 2022] DELETE /identity/v3/projects/1189d7a65d4346b7ab7b0302bbb5ec43 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:09.265786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d90a0-91a5-46a3-96fa-c1fb0c4ef0aa service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:09.278175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d90a0-91a5-46a3-96fa-c1fb0c4ef0aa service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:09.327533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:09.342339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:09.342975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:09.343255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:09.343557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] PATH_INFO: `/v3/users/2c058c15ca3a432eaee840384ae0548b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:09.344922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c058c15ca3a432eaee840384ae0548b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:09.346955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:09.409952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c058c15ca3a432eaee840384ae0548b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.420906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c058c15ca3a432eaee840384ae0548b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.421649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c058c15ca3a432eaee840384ae0548b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.434729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c058c15ca3a432eaee840384ae0548b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.446582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-391c59ab-adf6-4032-8cbb-cec28dd387ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c058c15ca3a432eaee840384ae0548b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:09.454635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1074/2152] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:09 2022] DELETE /identity/v3/users/2c058c15ca3a432eaee840384ae0548b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:09.971351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:09.973432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:09.991582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394bdc1-a939-457b-8a26-d3218ae2c1fa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:10.084552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:10.085208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.085478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.085743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] PATH_INFO: `/v3/projects/b0ac82d697db48c38e4cd023f5242668` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.087058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ac82d697db48c38e4cd023f5242668)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:10.089572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:10.138897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ac82d697db48c38e4cd023f5242668', 'request_id': 'req-7114591b-7013-4911-a34b-e9ab1ab9f776'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:10.144775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ac82d697db48c38e4cd023f5242668', 'request_id': 'req-7114591b-7013-4911-a34b-e9ab1ab9f776'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:10.152703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ac82d697db48c38e4cd023f5242668', 'request_id': 'req-7114591b-7013-4911-a34b-e9ab1ab9f776'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:10.163762 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] The token cache is being invalidate because project b0ac82d697db48c38e4cd023f5242668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:10.164042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:10.164674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7114591b-7013-4911-a34b-e9ab1ab9f776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:10.166141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1079/2153] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:09 2022] DELETE /identity/v3/projects/b0ac82d697db48c38e4cd023f5242668 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:10.171970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73f9e-5a03-49a7-be66-826b508c5d5c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.172261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73f9e-5a03-49a7-be66-826b508c5d5c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.172599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73f9e-5a03-49a7-be66-826b508c5d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.192796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7ea73f9e-5a03-49a7-be66-826b508c5d5c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:10.194029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7ea73f9e-5a03-49a7-be66-826b508c5d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:10.254523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1075/2154] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:10.262083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315971a-5b70-4d71-aad2-388f72289cd8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.262358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315971a-5b70-4d71-aad2-388f72289cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.262639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315971a-5b70-4d71-aad2-388f72289cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.283371 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6315971a-5b70-4d71-aad2-388f72289cd8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:10.284590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6315971a-5b70-4d71-aad2-388f72289cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:10.293127 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1080/2155] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:10.297242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea73f9e-5a03-49a7-be66-826b508c5d5c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:10.306934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea73f9e-5a03-49a7-be66-826b508c5d5c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:10.312169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d34ef-15d1-437b-b50f-2d46ce41407e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:10.312793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d34ef-15d1-437b-b50f-2d46ce41407e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.313080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d34ef-15d1-437b-b50f-2d46ce41407e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.313356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d34ef-15d1-437b-b50f-2d46ce41407e admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.314010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d34ef-15d1-437b-b50f-2d46ce41407e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:25:10.314316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d34ef-15d1-437b-b50f-2d46ce41407e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:10.315930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d34ef-15d1-437b-b50f-2d46ce41407e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:10.331076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1076/2156] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:25:10.332197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315971a-5b70-4d71-aad2-388f72289cd8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:10.341820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315971a-5b70-4d71-aad2-388f72289cd8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:10.346710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c74fae-f52a-430c-a81e-c3fe71dcf76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:10.347301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c74fae-f52a-430c-a81e-c3fe71dcf76f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.347582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c74fae-f52a-430c-a81e-c3fe71dcf76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.347856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c74fae-f52a-430c-a81e-c3fe71dcf76f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.348506 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c74fae-f52a-430c-a81e-c3fe71dcf76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:10.350430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c74fae-f52a-430c-a81e-c3fe71dcf76f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:10.382219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1081/2157] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:10 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:10.383605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d34ef-15d1-437b-b50f-2d46ce41407e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:10.393824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d34ef-15d1-437b-b50f-2d46ce41407e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:10.445068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:10.453384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a7477-2510-4c3b-b4cf-b8495f224077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:10.454189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a7477-2510-4c3b-b4cf-b8495f224077 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.454484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a7477-2510-4c3b-b4cf-b8495f224077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.455174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a7477-2510-4c3b-b4cf-b8495f224077 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.455759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a7477-2510-4c3b-b4cf-b8495f224077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:10.457749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a7477-2510-4c3b-b4cf-b8495f224077 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:10.483174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1077/2158] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:10 2022] POST /identity/v3/users => generated 279 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:10.486281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c74fae-f52a-430c-a81e-c3fe71dcf76f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:10.505865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c74fae-f52a-430c-a81e-c3fe71dcf76f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:10.511132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b6828-7ff5-4747-bfc8-54ebb8ed5518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:10.511811 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b6828-7ff5-4747-bfc8-54ebb8ed5518 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.512091 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b6828-7ff5-4747-bfc8-54ebb8ed5518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.512357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b6828-7ff5-4747-bfc8-54ebb8ed5518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.513085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b6828-7ff5-4747-bfc8-54ebb8ed5518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:10.514712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b6828-7ff5-4747-bfc8-54ebb8ed5518 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:10.530413 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1082/2159] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:10.534919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a7477-2510-4c3b-b4cf-b8495f224077 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:10.548371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a7477-2510-4c3b-b4cf-b8495f224077 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:10.555516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2b542-2f82-4a4c-9b50-0cec1dd265b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:10.556145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2b542-2f82-4a4c-9b50-0cec1dd265b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.556454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2b542-2f82-4a4c-9b50-0cec1dd265b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.556736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2b542-2f82-4a4c-9b50-0cec1dd265b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.557435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2b542-2f82-4a4c-9b50-0cec1dd265b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:10.559124 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2b542-2f82-4a4c-9b50-0cec1dd265b7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:10.573083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1078/2160] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:10.577773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b6828-7ff5-4747-bfc8-54ebb8ed5518 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:10.593097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b6828-7ff5-4747-bfc8-54ebb8ed5518 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:10.598232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5596df-95cf-4932-ac22-7049f637ebd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:10.598911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5596df-95cf-4932-ac22-7049f637ebd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.599188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5596df-95cf-4932-ac22-7049f637ebd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.599467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5596df-95cf-4932-ac22-7049f637ebd9 admin admin] PATH_INFO: `/v3/projects/603b1a0d3eb4412d944cb7d632317a87/users/9af5e162e7bd445e81c657e73076d66b/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.614710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:10.619763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5596df-95cf-4932-ac22-7049f637ebd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=603b1a0d3eb4412d944cb7d632317a87, user_id=9af5e162e7bd445e81c657e73076d66b, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:10.622831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5596df-95cf-4932-ac22-7049f637ebd9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:10.639480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1083/2161] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:10 2022] PUT /identity/v3/projects/603b1a0d3eb4412d944cb7d632317a87/users/9af5e162e7bd445e81c657e73076d66b/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:10.657550 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54fc58-584b-4cf8-ad6d-aa82f2584596 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.657836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54fc58-584b-4cf8-ad6d-aa82f2584596 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.658730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54fc58-584b-4cf8-ad6d-aa82f2584596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.712819 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6c54fc58-584b-4cf8-ad6d-aa82f2584596 None None] MFA Rules not processed for user `9af5e162e7bd445e81c657e73076d66b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:10.715223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6c54fc58-584b-4cf8-ad6d-aa82f2584596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:10.823412 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1079/2162] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:25:10.828115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5596df-95cf-4932-ac22-7049f637ebd9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:10.922622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5596df-95cf-4932-ac22-7049f637ebd9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:10.928304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7910e49-07c2-40f1-81f7-b2117a722822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:10.929084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7910e49-07c2-40f1-81f7-b2117a722822 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:10.929424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7910e49-07c2-40f1-81f7-b2117a722822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:10.929921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7910e49-07c2-40f1-81f7-b2117a722822 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:10.930578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7910e49-07c2-40f1-81f7-b2117a722822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:10.934712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7910e49-07c2-40f1-81f7-b2117a722822 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:10.987377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1084/2163] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:10 2022] POST /identity/v3/projects => generated 355 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:25:10.990611 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54fc58-584b-4cf8-ad6d-aa82f2584596 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:11.004478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54fc58-584b-4cf8-ad6d-aa82f2584596 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:11.102483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b306504-a12a-47b3-ba52-0d29bafad053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:11.103364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b306504-a12a-47b3-ba52-0d29bafad053 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:11.103688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b306504-a12a-47b3-ba52-0d29bafad053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:11.104215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b306504-a12a-47b3-ba52-0d29bafad053 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:11.105277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b306504-a12a-47b3-ba52-0d29bafad053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:11.108588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b306504-a12a-47b3-ba52-0d29bafad053 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:11.143494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1080/2164] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:10 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:11.145145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7910e49-07c2-40f1-81f7-b2117a722822 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:11.159352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7910e49-07c2-40f1-81f7-b2117a722822 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:11.164306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad201881-4e6b-4cc5-a8fe-337719e3881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:11.165030 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad201881-4e6b-4cc5-a8fe-337719e3881d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:11.165423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad201881-4e6b-4cc5-a8fe-337719e3881d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:11.165693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad201881-4e6b-4cc5-a8fe-337719e3881d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:11.166339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad201881-4e6b-4cc5-a8fe-337719e3881d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:11.168418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad201881-4e6b-4cc5-a8fe-337719e3881d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:11.180417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1085/2165] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:11.185823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b306504-a12a-47b3-ba52-0d29bafad053 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:11.202252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b306504-a12a-47b3-ba52-0d29bafad053 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:11.217577 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409169-3635-41dd-b00b-1aada297988f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:11.218266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409169-3635-41dd-b00b-1aada297988f admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:11.218549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409169-3635-41dd-b00b-1aada297988f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:11.218828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f409169-3635-41dd-b00b-1aada297988f admin admin] PATH_INFO: `/v3/projects/4c5d07aebd194f29b1b24b3b565775a9/users/bd3b599d1ef14e6daeb16bdeaf0df4e1/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:11.239666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f409169-3635-41dd-b00b-1aada297988f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5d07aebd194f29b1b24b3b565775a9, user_id=bd3b599d1ef14e6daeb16bdeaf0df4e1, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:11.242596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f409169-3635-41dd-b00b-1aada297988f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:11.258463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1081/2166] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:11 2022] PUT /identity/v3/projects/4c5d07aebd194f29b1b24b3b565775a9/users/bd3b599d1ef14e6daeb16bdeaf0df4e1/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:25:11.262765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad201881-4e6b-4cc5-a8fe-337719e3881d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:11.273520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad201881-4e6b-4cc5-a8fe-337719e3881d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:11.336664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2633159-c143-49f5-b484-783359a64390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:11.337364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2633159-c143-49f5-b484-783359a64390 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:11.337650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2633159-c143-49f5-b484-783359a64390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:11.337920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2633159-c143-49f5-b484-783359a64390 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:11.338597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2633159-c143-49f5-b484-783359a64390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:11.340788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2633159-c143-49f5-b484-783359a64390 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:11.352268 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1086/2167] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:11.357290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409169-3635-41dd-b00b-1aada297988f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:11.367639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f409169-3635-41dd-b00b-1aada297988f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:11.372657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222fa8a-cdd4-4fff-9801-111fb2438da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:11.373307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222fa8a-cdd4-4fff-9801-111fb2438da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:11.373584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222fa8a-cdd4-4fff-9801-111fb2438da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:11.373855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222fa8a-cdd4-4fff-9801-111fb2438da9 admin admin] PATH_INFO: `/v3/projects/4c5d07aebd194f29b1b24b3b565775a9/users/bd3b599d1ef14e6daeb16bdeaf0df4e1/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:11.375757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222fa8a-cdd4-4fff-9801-111fb2438da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5d07aebd194f29b1b24b3b565775a9, user_id=bd3b599d1ef14e6daeb16bdeaf0df4e1, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:11.378458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222fa8a-cdd4-4fff-9801-111fb2438da9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:11.389973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1082/2168] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:11 2022] PUT /identity/v3/projects/4c5d07aebd194f29b1b24b3b565775a9/users/bd3b599d1ef14e6daeb16bdeaf0df4e1/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:11.411511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a5e33-eeb9-4b39-b937-312b14b11e7d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:11.411794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a5e33-eeb9-4b39-b937-312b14b11e7d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:11.412069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a5e33-eeb9-4b39-b937-312b14b11e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:11.441949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ee5a5e33-eeb9-4b39-b937-312b14b11e7d None None] MFA Rules not processed for user `bd3b599d1ef14e6daeb16bdeaf0df4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:11.443201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ee5a5e33-eeb9-4b39-b937-312b14b11e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:11.493469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:11.516658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1087/2169] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:11.529244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222fa8a-cdd4-4fff-9801-111fb2438da9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:11.531060 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f222fa8a-cdd4-4fff-9801-111fb2438da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:11.550840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:11.593425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222fa8a-cdd4-4fff-9801-111fb2438da9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:11.597133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfea76-2814-4439-9257-876368687453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:11.597737 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfea76-2814-4439-9257-876368687453 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:11.598016 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfea76-2814-4439-9257-876368687453 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:11.598288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfea76-2814-4439-9257-876368687453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:11.608316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfea76-2814-4439-9257-876368687453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:11.609794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfea76-2814-4439-9257-876368687453 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:11.619327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1083/2170] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:11.698947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a5e33-eeb9-4b39-b937-312b14b11e7d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:11.708677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a5e33-eeb9-4b39-b937-312b14b11e7d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:11.712243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b544450-f8c8-46ec-b3d0-4d5bfefee034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:11.712898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b544450-f8c8-46ec-b3d0-4d5bfefee034 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:11.713178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b544450-f8c8-46ec-b3d0-4d5bfefee034 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:11.713449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b544450-f8c8-46ec-b3d0-4d5bfefee034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:11.772473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b544450-f8c8-46ec-b3d0-4d5bfefee034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:11.773864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b544450-f8c8-46ec-b3d0-4d5bfefee034 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:11.784120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1088/2171] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:18.865975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfea76-2814-4439-9257-876368687453 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:18.868711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-42cfea76-2814-4439-9257-876368687453 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:18.993318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfea76-2814-4439-9257-876368687453 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:18.998692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:18.999337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:18.999758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:18.999917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] PATH_INFO: `/v3/users/eebe100763294230bcf623d54cea890b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:19.001188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eebe100763294230bcf623d54cea890b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:19.005366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:19.071381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eebe100763294230bcf623d54cea890b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.081253 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eebe100763294230bcf623d54cea890b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.082214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eebe100763294230bcf623d54cea890b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.099256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eebe100763294230bcf623d54cea890b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.108511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eebe100763294230bcf623d54cea890b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.119476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1084/2172] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:18 2022] DELETE /identity/v3/users/eebe100763294230bcf623d54cea890b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:19.126769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e399c1d-bbfd-453a-916f-136ccec18786 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:19.128533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e399c1d-bbfd-453a-916f-136ccec18786 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:19.129163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e399c1d-bbfd-453a-916f-136ccec18786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:19.163596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8e399c1d-bbfd-453a-916f-136ccec18786 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:19.165373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e399c1d-bbfd-453a-916f-136ccec18786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:19.285210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1089/2173] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:25:19.296553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:19.298107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:19.351083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:19.384091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21058c5e-9f83-405f-9e9d-4004798cf49d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:19.389647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a285f-db9e-4528-afc3-3bb0a164d20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:19.390823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a285f-db9e-4528-afc3-3bb0a164d20d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:19.391125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a285f-db9e-4528-afc3-3bb0a164d20d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:19.391376 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a285f-db9e-4528-afc3-3bb0a164d20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:19.401308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a285f-db9e-4528-afc3-3bb0a164d20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:19.403510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a285f-db9e-4528-afc3-3bb0a164d20d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:19.413681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1085/2174] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:19.699641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e399c1d-bbfd-453a-916f-136ccec18786 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:19.701113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e399c1d-bbfd-453a-916f-136ccec18786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:19.714601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e399c1d-bbfd-453a-916f-136ccec18786 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:19.720545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:19.721350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:19.721685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:19.722022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] PATH_INFO: `/v3/projects/de8ee4e6cc2a4f49befed0b4c474349c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:19.723367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8ee4e6cc2a4f49befed0b4c474349c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:19.725702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:19.773832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8ee4e6cc2a4f49befed0b4c474349c', 'request_id': 'req-d5a01b46-34b6-4932-ae87-cf9c4b48a011'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.781333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8ee4e6cc2a4f49befed0b4c474349c', 'request_id': 'req-d5a01b46-34b6-4932-ae87-cf9c4b48a011'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.791103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8ee4e6cc2a4f49befed0b4c474349c', 'request_id': 'req-d5a01b46-34b6-4932-ae87-cf9c4b48a011'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.803918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] The token cache is being invalidate because project de8ee4e6cc2a4f49befed0b4c474349c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:19.804210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.804864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d5a01b46-34b6-4932-ae87-cf9c4b48a011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:19.806353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1090/2175] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:19 2022] DELETE /identity/v3/projects/de8ee4e6cc2a4f49befed0b4c474349c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:19.811794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7996-ee33-40a5-afe8-1f0fa55663be None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:19.812080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7996-ee33-40a5-afe8-1f0fa55663be None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:19.812357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7996-ee33-40a5-afe8-1f0fa55663be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:19.832209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-29be7996-ee33-40a5-afe8-1f0fa55663be None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:19.833472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-29be7996-ee33-40a5-afe8-1f0fa55663be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:19.896432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1086/2176] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:19.906218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ade5de-6d61-4b04-869b-373ab3191695 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:19.906503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ade5de-6d61-4b04-869b-373ab3191695 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:19.906780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ade5de-6d61-4b04-869b-373ab3191695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:19.921784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:19.927584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-72ade5de-6d61-4b04-869b-373ab3191695 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:19.929176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-72ade5de-6d61-4b04-869b-373ab3191695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:19.940783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1091/2177] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:25:19.945234 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be7996-ee33-40a5-afe8-1f0fa55663be None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:19.955705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be7996-ee33-40a5-afe8-1f0fa55663be None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:19.960237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753b720-c0da-41b7-986a-68e482cf3578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:19.960871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753b720-c0da-41b7-986a-68e482cf3578 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:19.961158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753b720-c0da-41b7-986a-68e482cf3578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:19.961435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753b720-c0da-41b7-986a-68e482cf3578 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:19.962085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753b720-c0da-41b7-986a-68e482cf3578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:25:19.962394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753b720-c0da-41b7-986a-68e482cf3578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:19.964766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753b720-c0da-41b7-986a-68e482cf3578 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:19.976640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1087/2178] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:25:19.981663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ade5de-6d61-4b04-869b-373ab3191695 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:19.992488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ade5de-6d61-4b04-869b-373ab3191695 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:19.997612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69bd36-ad9e-44b3-8eec-56b5937b802d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:19.998215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69bd36-ad9e-44b3-8eec-56b5937b802d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:19.998486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69bd36-ad9e-44b3-8eec-56b5937b802d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:19.998751 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69bd36-ad9e-44b3-8eec-56b5937b802d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:19.999371 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69bd36-ad9e-44b3-8eec-56b5937b802d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:20.001340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69bd36-ad9e-44b3-8eec-56b5937b802d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:20.036755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1092/2179] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:19 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:20.040904 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753b720-c0da-41b7-986a-68e482cf3578 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:20.052020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753b720-c0da-41b7-986a-68e482cf3578 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:20.118970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1787b32-dc5e-480a-87af-25eb36b177e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:20.119566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787b32-dc5e-480a-87af-25eb36b177e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:20.119844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787b32-dc5e-480a-87af-25eb36b177e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:20.120153 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1787b32-dc5e-480a-87af-25eb36b177e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:20.121147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1787b32-dc5e-480a-87af-25eb36b177e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:20.123625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1787b32-dc5e-480a-87af-25eb36b177e5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:20.147148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1088/2180] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:20 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:20.151255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69bd36-ad9e-44b3-8eec-56b5937b802d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:20.162648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69bd36-ad9e-44b3-8eec-56b5937b802d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:20.167349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f738ee8-9ad6-4dd7-a586-719b53b8ac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:20.168220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f738ee8-9ad6-4dd7-a586-719b53b8ac71 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:20.168519 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f738ee8-9ad6-4dd7-a586-719b53b8ac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:20.168793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f738ee8-9ad6-4dd7-a586-719b53b8ac71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:20.169438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f738ee8-9ad6-4dd7-a586-719b53b8ac71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:20.171057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f738ee8-9ad6-4dd7-a586-719b53b8ac71 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:20.184794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1093/2181] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:20.188040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1787b32-dc5e-480a-87af-25eb36b177e5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:20.203441 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1787b32-dc5e-480a-87af-25eb36b177e5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:20.209768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563f1ad-f82d-43c5-be05-3cfbabcf47b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:20.210385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563f1ad-f82d-43c5-be05-3cfbabcf47b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:20.210667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563f1ad-f82d-43c5-be05-3cfbabcf47b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:20.210943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563f1ad-f82d-43c5-be05-3cfbabcf47b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:20.211598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563f1ad-f82d-43c5-be05-3cfbabcf47b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:20.213833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563f1ad-f82d-43c5-be05-3cfbabcf47b7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:20.232185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1089/2182] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:20.236768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f738ee8-9ad6-4dd7-a586-719b53b8ac71 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:20.249932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f738ee8-9ad6-4dd7-a586-719b53b8ac71 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:20.255065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cd000-f07a-4baa-9c1b-d3306a998429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:20.255843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cd000-f07a-4baa-9c1b-d3306a998429 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:20.256216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cd000-f07a-4baa-9c1b-d3306a998429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:20.256622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cd000-f07a-4baa-9c1b-d3306a998429 admin admin] PATH_INFO: `/v3/projects/b4787a35fb004ea6beee8cc95f4136c8/users/2c575afe513548bfbfe379d207952ef0/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:20.273517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cd000-f07a-4baa-9c1b-d3306a998429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4787a35fb004ea6beee8cc95f4136c8, user_id=2c575afe513548bfbfe379d207952ef0, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:20.276074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cd000-f07a-4baa-9c1b-d3306a998429 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:20.293151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1094/2183] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:20 2022] PUT /identity/v3/projects/b4787a35fb004ea6beee8cc95f4136c8/users/2c575afe513548bfbfe379d207952ef0/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:20.304090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb4889-6f44-4dbe-a3f3-a5373c8b3867 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:20.304405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb4889-6f44-4dbe-a3f3-a5373c8b3867 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:20.304688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb4889-6f44-4dbe-a3f3-a5373c8b3867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:20.340024 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2eeb4889-6f44-4dbe-a3f3-a5373c8b3867 None None] MFA Rules not processed for user `2c575afe513548bfbfe379d207952ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:20.341596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2eeb4889-6f44-4dbe-a3f3-a5373c8b3867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:20.391601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:20.440634 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1090/2184] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:20.452026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cd000-f07a-4baa-9c1b-d3306a998429 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:20.454101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-273cd000-f07a-4baa-9c1b-d3306a998429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:20.539558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cd000-f07a-4baa-9c1b-d3306a998429 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:20.546053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86ae9a-b70d-46ce-8f93-032a52614d2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:20.547485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86ae9a-b70d-46ce-8f93-032a52614d2a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:20.547485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86ae9a-b70d-46ce-8f93-032a52614d2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:20.547650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86ae9a-b70d-46ce-8f93-032a52614d2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:20.570221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86ae9a-b70d-46ce-8f93-032a52614d2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:20.571915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86ae9a-b70d-46ce-8f93-032a52614d2a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:20.586843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1095/2185] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:20.630359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb4889-6f44-4dbe-a3f3-a5373c8b3867 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:20.725902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb4889-6f44-4dbe-a3f3-a5373c8b3867 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:20.730354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:20.730973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:20.731249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:20.731535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] PATH_INFO: `/v3/users/2c575afe513548bfbfe379d207952ef0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:20.732664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c575afe513548bfbfe379d207952ef0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:20.734584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:20.784567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c575afe513548bfbfe379d207952ef0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:20.795687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c575afe513548bfbfe379d207952ef0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:20.797091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c575afe513548bfbfe379d207952ef0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:20.817081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c575afe513548bfbfe379d207952ef0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:20.828316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c575afe513548bfbfe379d207952ef0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:20.835273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1091/2186] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:20 2022] DELETE /identity/v3/users/2c575afe513548bfbfe379d207952ef0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:20.841979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07b982-8d8b-43dc-975d-a84b6bbebd08 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:20.842314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07b982-8d8b-43dc-975d-a84b6bbebd08 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:20.842627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07b982-8d8b-43dc-975d-a84b6bbebd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:20.862700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2d07b982-8d8b-43dc-975d-a84b6bbebd08 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:20.863929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d07b982-8d8b-43dc-975d-a84b6bbebd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:20.926850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1096/2187] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:20.936651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:20.938199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:20.997505 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ba54a-289a-4dd5-bfe3-8a7b75a32619 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:21.001239 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80c098-9989-4f78-91ec-0d939dba3032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:21.001847 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80c098-9989-4f78-91ec-0d939dba3032 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.002125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80c098-9989-4f78-91ec-0d939dba3032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.002397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80c098-9989-4f78-91ec-0d939dba3032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.012809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80c098-9989-4f78-91ec-0d939dba3032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:21.014252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80c098-9989-4f78-91ec-0d939dba3032 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:21.018327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:21.026088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1092/2188] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:21.308473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07b982-8d8b-43dc-975d-a84b6bbebd08 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:21.310725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d07b982-8d8b-43dc-975d-a84b6bbebd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:21.326836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07b982-8d8b-43dc-975d-a84b6bbebd08 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:21.332051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:21.332890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.333172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.333456 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] PATH_INFO: `/v3/projects/b4787a35fb004ea6beee8cc95f4136c8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.334897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4787a35fb004ea6beee8cc95f4136c8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:21.336961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:21.383419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:21.392364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4787a35fb004ea6beee8cc95f4136c8', 'request_id': 'req-4a35da2e-22d1-47df-9f1c-fae72de9e49a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:21.399046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4787a35fb004ea6beee8cc95f4136c8', 'request_id': 'req-4a35da2e-22d1-47df-9f1c-fae72de9e49a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:21.409533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4787a35fb004ea6beee8cc95f4136c8', 'request_id': 'req-4a35da2e-22d1-47df-9f1c-fae72de9e49a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:21.421810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] The token cache is being invalidate because project b4787a35fb004ea6beee8cc95f4136c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:21.422104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:21.422761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a35da2e-22d1-47df-9f1c-fae72de9e49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:21.424970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1097/2189] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:21 2022] DELETE /identity/v3/projects/b4787a35fb004ea6beee8cc95f4136c8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:21.429582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af378-af2a-4e26-aaf6-8154743a7e74 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.429582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af378-af2a-4e26-aaf6-8154743a7e74 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.429582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af378-af2a-4e26-aaf6-8154743a7e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.458479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5c5af378-af2a-4e26-aaf6-8154743a7e74 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:21.459712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5c5af378-af2a-4e26-aaf6-8154743a7e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:21.529242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1093/2190] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:21.537819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c861a75-a8a2-4089-91e9-80ad19e83560 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.538101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c861a75-a8a2-4089-91e9-80ad19e83560 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.538377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c861a75-a8a2-4089-91e9-80ad19e83560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.561339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3c861a75-a8a2-4089-91e9-80ad19e83560 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:21.562560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3c861a75-a8a2-4089-91e9-80ad19e83560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:21.570479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1098/2191] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:21.575704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5af378-af2a-4e26-aaf6-8154743a7e74 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:21.588764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5af378-af2a-4e26-aaf6-8154743a7e74 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:21.594088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5318291-1db7-4746-ad6e-181a1f7f30e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:21.594696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5318291-1db7-4746-ad6e-181a1f7f30e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.594975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5318291-1db7-4746-ad6e-181a1f7f30e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.595245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5318291-1db7-4746-ad6e-181a1f7f30e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.595896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5318291-1db7-4746-ad6e-181a1f7f30e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:25:21.596196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5318291-1db7-4746-ad6e-181a1f7f30e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:21.597887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5318291-1db7-4746-ad6e-181a1f7f30e5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:21.611317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1094/2192] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:25:21.616101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c861a75-a8a2-4089-91e9-80ad19e83560 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:21.626673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c861a75-a8a2-4089-91e9-80ad19e83560 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:21.631418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c62b5-936a-404c-8c01-f63e99cf60ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:21.632014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c62b5-936a-404c-8c01-f63e99cf60ae admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.632291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c62b5-936a-404c-8c01-f63e99cf60ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.632590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c62b5-936a-404c-8c01-f63e99cf60ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.633227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c62b5-936a-404c-8c01-f63e99cf60ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:21.635184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c62b5-936a-404c-8c01-f63e99cf60ae admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:21.665944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1099/2193] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:21 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:21.672976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5318291-1db7-4746-ad6e-181a1f7f30e5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:21.687528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5318291-1db7-4746-ad6e-181a1f7f30e5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:21.760433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55bafb-8afc-467c-ba17-3e9f47ede5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:21.761939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55bafb-8afc-467c-ba17-3e9f47ede5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.761939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55bafb-8afc-467c-ba17-3e9f47ede5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.761939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55bafb-8afc-467c-ba17-3e9f47ede5c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.762437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55bafb-8afc-467c-ba17-3e9f47ede5c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:21.764756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55bafb-8afc-467c-ba17-3e9f47ede5c4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:21.796720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1095/2194] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:21 2022] POST /identity/v3/users => generated 300 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:21.797931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c62b5-936a-404c-8c01-f63e99cf60ae admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:21.809778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c62b5-936a-404c-8c01-f63e99cf60ae admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:21.815202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf200acb-6c8d-4ef0-80fc-69542d9bea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:21.815844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf200acb-6c8d-4ef0-80fc-69542d9bea3c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.816129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf200acb-6c8d-4ef0-80fc-69542d9bea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.816441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf200acb-6c8d-4ef0-80fc-69542d9bea3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.817098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf200acb-6c8d-4ef0-80fc-69542d9bea3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:21.818733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf200acb-6c8d-4ef0-80fc-69542d9bea3c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:21.833320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1100/2195] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:21.834739 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55bafb-8afc-467c-ba17-3e9f47ede5c4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:21.845482 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55bafb-8afc-467c-ba17-3e9f47ede5c4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:21.850371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b062d40-5ba5-4b85-a4f0-1b55169853e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:21.850986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b062d40-5ba5-4b85-a4f0-1b55169853e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.851267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b062d40-5ba5-4b85-a4f0-1b55169853e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.851539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b062d40-5ba5-4b85-a4f0-1b55169853e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.852190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b062d40-5ba5-4b85-a4f0-1b55169853e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:21.853882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b062d40-5ba5-4b85-a4f0-1b55169853e4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:21.865951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1096/2196] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:21.870299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf200acb-6c8d-4ef0-80fc-69542d9bea3c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:21.880600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf200acb-6c8d-4ef0-80fc-69542d9bea3c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:21.885286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44247cbe-e8c3-4fa5-a505-8ae0ece9542a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:21.886377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44247cbe-e8c3-4fa5-a505-8ae0ece9542a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.886659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44247cbe-e8c3-4fa5-a505-8ae0ece9542a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.886931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44247cbe-e8c3-4fa5-a505-8ae0ece9542a admin admin] PATH_INFO: `/v3/projects/ee5d3b4d51244d618739db4398ffdcd6/users/8bf3c878466a491a8fad71d11d1ba1f1/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.902257 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44247cbe-e8c3-4fa5-a505-8ae0ece9542a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee5d3b4d51244d618739db4398ffdcd6, user_id=8bf3c878466a491a8fad71d11d1ba1f1, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:21.904982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44247cbe-e8c3-4fa5-a505-8ae0ece9542a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:21.918746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1101/2197] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:21 2022] PUT /identity/v3/projects/ee5d3b4d51244d618739db4398ffdcd6/users/8bf3c878466a491a8fad71d11d1ba1f1/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:21.930094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440bb97-739b-46a0-934d-7557abb34254 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:21.930314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440bb97-739b-46a0-934d-7557abb34254 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:21.930588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440bb97-739b-46a0-934d-7557abb34254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:21.958745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a440bb97-739b-46a0-934d-7557abb34254 None None] MFA Rules not processed for user `8bf3c878466a491a8fad71d11d1ba1f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:21.959959 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a440bb97-739b-46a0-934d-7557abb34254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:22.014484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:22.024720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1097/2198] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:21 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:25:22.036695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44247cbe-e8c3-4fa5-a505-8ae0ece9542a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:22.038437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-44247cbe-e8c3-4fa5-a505-8ae0ece9542a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:22.096129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44247cbe-e8c3-4fa5-a505-8ae0ece9542a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:22.099758 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61e7f7-f447-49d0-adf5-5c3aca9d54e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:22.100344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e7f7-f447-49d0-adf5-5c3aca9d54e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:22.100726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e7f7-f447-49d0-adf5-5c3aca9d54e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:22.100998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61e7f7-f447-49d0-adf5-5c3aca9d54e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:22.109912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61e7f7-f447-49d0-adf5-5c3aca9d54e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:22.111669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61e7f7-f447-49d0-adf5-5c3aca9d54e0 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:22.120561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1102/2199] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:22 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:23.445826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772949b3-15d7-4d50-8692-045093ac8e20 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:23.446376 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772949b3-15d7-4d50-8692-045093ac8e20 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:23.446376 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772949b3-15d7-4d50-8692-045093ac8e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:23.468104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-772949b3-15d7-4d50-8692-045093ac8e20 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:23.469356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-772949b3-15d7-4d50-8692-045093ac8e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:23.547173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1098/2200] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:25:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:23.620288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61e7f7-f447-49d0-adf5-5c3aca9d54e0 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:23.621943 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0f61e7f7-f447-49d0-adf5-5c3aca9d54e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:23.709022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61e7f7-f447-49d0-adf5-5c3aca9d54e0 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:23.715089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baf069-55fa-431f-b402-fed09a44b8ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:23.715740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baf069-55fa-431f-b402-fed09a44b8ed service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:23.716019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baf069-55fa-431f-b402-fed09a44b8ed service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:23.716281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baf069-55fa-431f-b402-fed09a44b8ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:23.731320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baf069-55fa-431f-b402-fed09a44b8ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:23.733041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baf069-55fa-431f-b402-fed09a44b8ed service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:23.736985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:23.744334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1103/2201] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:25:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 15:25:29.816995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb6e5f-5f4f-4967-9a08-cfcd56049a9a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:29.817621 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb6e5f-5f4f-4967-9a08-cfcd56049a9a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:29.817932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb6e5f-5f4f-4967-9a08-cfcd56049a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:29.841305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-58cb6e5f-5f4f-4967-9a08-cfcd56049a9a None None] MFA Rules not processed for user `218f3dd17e6549fc8796eb30e4b48e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:29.842725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-58cb6e5f-5f4f-4967-9a08-cfcd56049a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:29.925156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1099/2202] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:30.620757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baf069-55fa-431f-b402-fed09a44b8ed service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:30.696367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baf069-55fa-431f-b402-fed09a44b8ed service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:30.703714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:30.704365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:30.704675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:30.704967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] PATH_INFO: `/v3/users/8bf3c878466a491a8fad71d11d1ba1f1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:30.706136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf3c878466a491a8fad71d11d1ba1f1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:30.708164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:30.777338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf3c878466a491a8fad71d11d1ba1f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:30.778180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf3c878466a491a8fad71d11d1ba1f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:30.779710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf3c878466a491a8fad71d11d1ba1f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:30.789587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf3c878466a491a8fad71d11d1ba1f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:30.806334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf3c878466a491a8fad71d11d1ba1f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:30.836035 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1104/2203] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:30 2022] DELETE /identity/v3/users/8bf3c878466a491a8fad71d11d1ba1f1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:30.842023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6361c-8977-4972-80af-75b46d4471bb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:30.842583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6361c-8977-4972-80af-75b46d4471bb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:30.842768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6361c-8977-4972-80af-75b46d4471bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:30.865804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a3c6361c-8977-4972-80af-75b46d4471bb None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:30.867303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a3c6361c-8977-4972-80af-75b46d4471bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:30.961972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1100/2204] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:25:30.976331 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:30.978054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:31.060684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6fc8c-cd87-43ac-aea8-6dd6c19c9e4b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:31.072045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4412d6f-193e-4915-afe4-8159c6000850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:31.073054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4412d6f-193e-4915-afe4-8159c6000850 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:31.074517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4412d6f-193e-4915-afe4-8159c6000850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:31.075439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4412d6f-193e-4915-afe4-8159c6000850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:31.089772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4412d6f-193e-4915-afe4-8159c6000850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:31.092083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4412d6f-193e-4915-afe4-8159c6000850 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:31.104846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1105/2205] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:31.585008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6361c-8977-4972-80af-75b46d4471bb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:31.586683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a3c6361c-8977-4972-80af-75b46d4471bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:31.602988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6361c-8977-4972-80af-75b46d4471bb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:31.613228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:31.614133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:31.614962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:31.615227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] PATH_INFO: `/v3/projects/ee5d3b4d51244d618739db4398ffdcd6` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:31.616795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee5d3b4d51244d618739db4398ffdcd6)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:31.618469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:31.664188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:31.674660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee5d3b4d51244d618739db4398ffdcd6', 'request_id': 'req-9c9b5733-8362-416d-bc26-8f375b2a1926'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:31.684983 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee5d3b4d51244d618739db4398ffdcd6', 'request_id': 'req-9c9b5733-8362-416d-bc26-8f375b2a1926'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:31.693253 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee5d3b4d51244d618739db4398ffdcd6', 'request_id': 'req-9c9b5733-8362-416d-bc26-8f375b2a1926'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:31.706313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] The token cache is being invalidate because project ee5d3b4d51244d618739db4398ffdcd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:31.706688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:31.707413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9c9b5733-8362-416d-bc26-8f375b2a1926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:31.709132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1101/2206] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:31 2022] DELETE /identity/v3/projects/ee5d3b4d51244d618739db4398ffdcd6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:31.715326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7025e-7d37-48ff-b394-c8df242289e4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:31.716111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7025e-7d37-48ff-b394-c8df242289e4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:31.716111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7025e-7d37-48ff-b394-c8df242289e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:31.752312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-66e7025e-7d37-48ff-b394-c8df242289e4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:31.754123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66e7025e-7d37-48ff-b394-c8df242289e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:31.785507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:31.875380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1106/2207] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:31.878705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f7f03-fa33-4b4e-bae4-686c17ef6cb2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:31.879283 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f7f03-fa33-4b4e-bae4-686c17ef6cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:31.879480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f7f03-fa33-4b4e-bae4-686c17ef6cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:31.902160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-457f7f03-fa33-4b4e-bae4-686c17ef6cb2 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:31.903599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-457f7f03-fa33-4b4e-bae4-686c17ef6cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:31.923197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1102/2208] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:25:31.930066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7025e-7d37-48ff-b394-c8df242289e4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:31.952130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7025e-7d37-48ff-b394-c8df242289e4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:31.982378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04aa0d-bf08-4588-8259-eab6653f4af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:31.983209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04aa0d-bf08-4588-8259-eab6653f4af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:31.984352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04aa0d-bf08-4588-8259-eab6653f4af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:31.984740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04aa0d-bf08-4588-8259-eab6653f4af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:31.989621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04aa0d-bf08-4588-8259-eab6653f4af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:25:31.990295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04aa0d-bf08-4588-8259-eab6653f4af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:31.992363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04aa0d-bf08-4588-8259-eab6653f4af6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:32.007209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1107/2209] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:25:32.012319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f7f03-fa33-4b4e-bae4-686c17ef6cb2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:32.023835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f7f03-fa33-4b4e-bae4-686c17ef6cb2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:32.030380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afb08b-dcf0-4ac5-906c-df09672e9140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:32.031158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afb08b-dcf0-4ac5-906c-df09672e9140 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:32.031534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afb08b-dcf0-4ac5-906c-df09672e9140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:32.032302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afb08b-dcf0-4ac5-906c-df09672e9140 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:32.032784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afb08b-dcf0-4ac5-906c-df09672e9140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:32.034981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afb08b-dcf0-4ac5-906c-df09672e9140 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:32.079131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1103/2210] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:32 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:32.083353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04aa0d-bf08-4588-8259-eab6653f4af6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:32.096151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04aa0d-bf08-4588-8259-eab6653f4af6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:32.171957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594fb29-3db0-416e-a870-31f1338703f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:32.172664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594fb29-3db0-416e-a870-31f1338703f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:32.173022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594fb29-3db0-416e-a870-31f1338703f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:32.173375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594fb29-3db0-416e-a870-31f1338703f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:32.174257 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594fb29-3db0-416e-a870-31f1338703f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:32.176314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594fb29-3db0-416e-a870-31f1338703f0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:32.208820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1108/2211] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:32 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:32.212925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afb08b-dcf0-4ac5-906c-df09672e9140 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:32.224508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afb08b-dcf0-4ac5-906c-df09672e9140 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:32.230387 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f43ca-ed55-4e32-bc92-8d1bd52e9971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:32.231264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f43ca-ed55-4e32-bc92-8d1bd52e9971 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:32.231543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f43ca-ed55-4e32-bc92-8d1bd52e9971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:32.231860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f43ca-ed55-4e32-bc92-8d1bd52e9971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:32.232559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f43ca-ed55-4e32-bc92-8d1bd52e9971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:32.234171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f43ca-ed55-4e32-bc92-8d1bd52e9971 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:32.248438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1104/2212] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:32.250656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594fb29-3db0-416e-a870-31f1338703f0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:32.261745 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594fb29-3db0-416e-a870-31f1338703f0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:32.265901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba79a6-f447-4920-8abd-cd2bba7fc019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:32.266531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba79a6-f447-4920-8abd-cd2bba7fc019 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:32.266803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba79a6-f447-4920-8abd-cd2bba7fc019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:32.267070 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba79a6-f447-4920-8abd-cd2bba7fc019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:32.267819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba79a6-f447-4920-8abd-cd2bba7fc019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:32.269451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba79a6-f447-4920-8abd-cd2bba7fc019 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:32.282226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1109/2213] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:25:32.286386 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f43ca-ed55-4e32-bc92-8d1bd52e9971 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:32.297937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f43ca-ed55-4e32-bc92-8d1bd52e9971 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:32.303759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4ec18-cd6b-4220-b39a-b8265cf0f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:32.304790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ec18-cd6b-4220-b39a-b8265cf0f462 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:32.305006 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ec18-cd6b-4220-b39a-b8265cf0f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:32.305242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4ec18-cd6b-4220-b39a-b8265cf0f462 admin admin] PATH_INFO: `/v3/projects/0add00db6668475093f558e9700333d1/users/af3e7eea0b2840b79404b370382ab44a/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:32.321697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4ec18-cd6b-4220-b39a-b8265cf0f462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0add00db6668475093f558e9700333d1, user_id=af3e7eea0b2840b79404b370382ab44a, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:32.325249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4ec18-cd6b-4220-b39a-b8265cf0f462 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:32.339493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1105/2214] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:32 2022] PUT /identity/v3/projects/0add00db6668475093f558e9700333d1/users/af3e7eea0b2840b79404b370382ab44a/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:32.344612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5bf4d-4740-46b6-ad0d-971e34405e20 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:32.344935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5bf4d-4740-46b6-ad0d-971e34405e20 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:32.345180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5bf4d-4740-46b6-ad0d-971e34405e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:32.366545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fcd5bf4d-4740-46b6-ad0d-971e34405e20 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:32.367766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fcd5bf4d-4740-46b6-ad0d-971e34405e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:32.488884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1110/2215] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:32.503159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4ec18-cd6b-4220-b39a-b8265cf0f462 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:32.505170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-24d4ec18-cd6b-4220-b39a-b8265cf0f462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:32.650738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4ec18-cd6b-4220-b39a-b8265cf0f462 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:32.657670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bef7c9-bedb-4087-966b-2fa52863911b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:32.658809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef7c9-bedb-4087-966b-2fa52863911b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:32.659442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef7c9-bedb-4087-966b-2fa52863911b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:32.665982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef7c9-bedb-4087-966b-2fa52863911b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:32.700434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bef7c9-bedb-4087-966b-2fa52863911b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:32.702240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bef7c9-bedb-4087-966b-2fa52863911b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:32.715020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1106/2216] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:32.966224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5bf4d-4740-46b6-ad0d-971e34405e20 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:32.967938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fcd5bf4d-4740-46b6-ad0d-971e34405e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:32.987719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5bf4d-4740-46b6-ad0d-971e34405e20 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:32.997558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:32.998185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:32.999115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:32.999672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] PATH_INFO: `/v3/users/9af5e162e7bd445e81c657e73076d66b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:33.001543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af5e162e7bd445e81c657e73076d66b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:33.005679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:33.036549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:33.084465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af5e162e7bd445e81c657e73076d66b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:33.085284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af5e162e7bd445e81c657e73076d66b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:33.085920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af5e162e7bd445e81c657e73076d66b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:33.097432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af5e162e7bd445e81c657e73076d66b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:33.116188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af5e162e7bd445e81c657e73076d66b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:33.138661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1111/2217] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:32 2022] DELETE /identity/v3/users/9af5e162e7bd445e81c657e73076d66b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:33.145120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8b9ae-4e76-4feb-9f1a-ec5f7f50bb91 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:33.146211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8b9ae-4e76-4feb-9f1a-ec5f7f50bb91 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:33.146714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8b9ae-4e76-4feb-9f1a-ec5f7f50bb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:33.175025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7af8b9ae-4e76-4feb-9f1a-ec5f7f50bb91 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:33.176494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7af8b9ae-4e76-4feb-9f1a-ec5f7f50bb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:33.296289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1107/2218] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:25:33.302213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:33.303890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:33.392598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49281859-91af-4515-88b7-b402627d3bed admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:33.396777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894de9a-4192-49fc-b4db-5eca28782d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:33.397559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894de9a-4192-49fc-b4db-5eca28782d1a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:33.397942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894de9a-4192-49fc-b4db-5eca28782d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:33.398840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894de9a-4192-49fc-b4db-5eca28782d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:33.417419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894de9a-4192-49fc-b4db-5eca28782d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:33.419338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894de9a-4192-49fc-b4db-5eca28782d1a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:33.433548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1112/2219] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:34.073117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8b9ae-4e76-4feb-9f1a-ec5f7f50bb91 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:34.075437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7af8b9ae-4e76-4feb-9f1a-ec5f7f50bb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:34.113542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8b9ae-4e76-4feb-9f1a-ec5f7f50bb91 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:34.143537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:34.144053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:34.144576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:34.145132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] PATH_INFO: `/v3/projects/603b1a0d3eb4412d944cb7d632317a87` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:34.147868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=603b1a0d3eb4412d944cb7d632317a87)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:34.150912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:34.155417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:34.285318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '603b1a0d3eb4412d944cb7d632317a87', 'request_id': 'req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.306943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '603b1a0d3eb4412d944cb7d632317a87', 'request_id': 'req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.321719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '603b1a0d3eb4412d944cb7d632317a87', 'request_id': 'req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.351267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] The token cache is being invalidate because project 603b1a0d3eb4412d944cb7d632317a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:34.351732 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.353371 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.358851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1108/2220] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:34 2022] DELETE /identity/v3/projects/603b1a0d3eb4412d944cb7d632317a87 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:34.360684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894de9a-4192-49fc-b4db-5eca28782d1a service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:34.377572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894de9a-4192-49fc-b4db-5eca28782d1a service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:34.523000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:34.523741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:34.524133 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:34.524434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] PATH_INFO: `/v3/users/bd3b599d1ef14e6daeb16bdeaf0df4e1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:34.527532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd3b599d1ef14e6daeb16bdeaf0df4e1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:34.531084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:34.644784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:34.648072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd3b599d1ef14e6daeb16bdeaf0df4e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.649538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd3b599d1ef14e6daeb16bdeaf0df4e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.650974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd3b599d1ef14e6daeb16bdeaf0df4e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.663431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd3b599d1ef14e6daeb16bdeaf0df4e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.682668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ac66c28e-8e1a-4ece-bd52-248eea1f33b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd3b599d1ef14e6daeb16bdeaf0df4e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:34.724859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1113/2221] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:34 2022] DELETE /identity/v3/users/bd3b599d1ef14e6daeb16bdeaf0df4e1 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:35.265541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:35.267615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:35.289981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac40aa-1c45-4ef4-8c7a-35cdfa7c9602 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:35.463521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:35.465923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:35.467290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:35.467819 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] PATH_INFO: `/v3/projects/4c5d07aebd194f29b1b24b3b565775a9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:35.469563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5d07aebd194f29b1b24b3b565775a9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:35.473763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:35.596114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5d07aebd194f29b1b24b3b565775a9', 'request_id': 'req-fbb95c86-d6cb-468a-a5d6-97883226705a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:35.619841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5d07aebd194f29b1b24b3b565775a9', 'request_id': 'req-fbb95c86-d6cb-468a-a5d6-97883226705a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:35.651964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5d07aebd194f29b1b24b3b565775a9', 'request_id': 'req-fbb95c86-d6cb-468a-a5d6-97883226705a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:35.678163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] The token cache is being invalidate because project 4c5d07aebd194f29b1b24b3b565775a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:35.678511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:35.689154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbb95c86-d6cb-468a-a5d6-97883226705a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:35.693364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1109/2222] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:35 2022] DELETE /identity/v3/projects/4c5d07aebd194f29b1b24b3b565775a9 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:35.699000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf86db-afae-473b-96b2-b670558ea970 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:35.699545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf86db-afae-473b-96b2-b670558ea970 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:35.700300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf86db-afae-473b-96b2-b670558ea970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:35.748337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8bdf86db-afae-473b-96b2-b670558ea970 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:35.750243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8bdf86db-afae-473b-96b2-b670558ea970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:35.852282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1114/2223] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:35.865271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8399d29-692b-4866-9b06-9fc5381e4974 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:35.865736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8399d29-692b-4866-9b06-9fc5381e4974 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:35.866020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8399d29-692b-4866-9b06-9fc5381e4974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:35.911214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f8399d29-692b-4866-9b06-9fc5381e4974 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:35.913846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f8399d29-692b-4866-9b06-9fc5381e4974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:35.937804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1110/2224] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:35.943466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf86db-afae-473b-96b2-b670558ea970 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:35.969218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf86db-afae-473b-96b2-b670558ea970 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:35.977349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84b057-7719-4442-ba33-69c32f780e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:35.978445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84b057-7719-4442-ba33-69c32f780e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:35.979114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84b057-7719-4442-ba33-69c32f780e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:35.980600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84b057-7719-4442-ba33-69c32f780e0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:35.982138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84b057-7719-4442-ba33-69c32f780e0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:25:35.982813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84b057-7719-4442-ba33-69c32f780e0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:35.986245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84b057-7719-4442-ba33-69c32f780e0e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:36.012321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1115/2225] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:25:36.017702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8399d29-692b-4866-9b06-9fc5381e4974 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:36.035884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8399d29-692b-4866-9b06-9fc5381e4974 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:36.053855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273a2e8-b6f1-40a5-86eb-2e415836ac1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:36.054499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273a2e8-b6f1-40a5-86eb-2e415836ac1f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:36.054779 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273a2e8-b6f1-40a5-86eb-2e415836ac1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:36.055727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273a2e8-b6f1-40a5-86eb-2e415836ac1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:36.055727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273a2e8-b6f1-40a5-86eb-2e415836ac1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:36.060687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273a2e8-b6f1-40a5-86eb-2e415836ac1f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:36.111558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:36.141947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1111/2226] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:36 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:36.143762 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84b057-7719-4442-ba33-69c32f780e0e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:36.171307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84b057-7719-4442-ba33-69c32f780e0e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:36.332365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05da877-a41e-4bff-84ff-8958beac52aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:36.333018 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05da877-a41e-4bff-84ff-8958beac52aa admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:36.333440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05da877-a41e-4bff-84ff-8958beac52aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:36.333718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05da877-a41e-4bff-84ff-8958beac52aa admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:36.334359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05da877-a41e-4bff-84ff-8958beac52aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:36.336593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05da877-a41e-4bff-84ff-8958beac52aa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:36.394786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273a2e8-b6f1-40a5-86eb-2e415836ac1f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:36.403526 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1116/2227] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:36 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:36.438995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273a2e8-b6f1-40a5-86eb-2e415836ac1f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:36.455992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d0c50-9460-4689-8caa-585105906393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:36.456953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d0c50-9460-4689-8caa-585105906393 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:36.460681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d0c50-9460-4689-8caa-585105906393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:36.461057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d0c50-9460-4689-8caa-585105906393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:36.463997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d0c50-9460-4689-8caa-585105906393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:36.466306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d0c50-9460-4689-8caa-585105906393 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:36.502110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05da877-a41e-4bff-84ff-8958beac52aa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:36.522406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1112/2228] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:36.529121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05da877-a41e-4bff-84ff-8958beac52aa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:36.536812 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337034c1-847b-48fe-8baf-1f365795ee7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:36.537446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337034c1-847b-48fe-8baf-1f365795ee7d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:36.537747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337034c1-847b-48fe-8baf-1f365795ee7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:36.538108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337034c1-847b-48fe-8baf-1f365795ee7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:36.538848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337034c1-847b-48fe-8baf-1f365795ee7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:36.541337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337034c1-847b-48fe-8baf-1f365795ee7d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:36.571399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1117/2229] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:36.577657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d0c50-9460-4689-8caa-585105906393 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:36.592106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d0c50-9460-4689-8caa-585105906393 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:36.599114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990c9fc-8917-4def-9e54-3fd180782b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:36.599852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990c9fc-8917-4def-9e54-3fd180782b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:36.600136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990c9fc-8917-4def-9e54-3fd180782b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:36.600429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990c9fc-8917-4def-9e54-3fd180782b06 admin admin] PATH_INFO: `/v3/projects/06d64e6f093249f99bb85493be3ca1a5/users/5d61454f9f464e22938a1ab9a9c603ae/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:36.635370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990c9fc-8917-4def-9e54-3fd180782b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d64e6f093249f99bb85493be3ca1a5, user_id=5d61454f9f464e22938a1ab9a9c603ae, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:36.641301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990c9fc-8917-4def-9e54-3fd180782b06 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:36.683451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1113/2230] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:36 2022] PUT /identity/v3/projects/06d64e6f093249f99bb85493be3ca1a5/users/5d61454f9f464e22938a1ab9a9c603ae/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:36.685854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac306e4-7505-4b62-9e3b-359be7617c15 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:36.686342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac306e4-7505-4b62-9e3b-359be7617c15 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:36.686788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac306e4-7505-4b62-9e3b-359be7617c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:36.737679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1ac306e4-7505-4b62-9e3b-359be7617c15 None None] MFA Rules not processed for user `5d61454f9f464e22938a1ab9a9c603ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:36.740697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1ac306e4-7505-4b62-9e3b-359be7617c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:36.775454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:36.901995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1118/2231] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:36.903923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990c9fc-8917-4def-9e54-3fd180782b06 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:37.046812 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990c9fc-8917-4def-9e54-3fd180782b06 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:37.054919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff208-3224-46d8-9de9-81656e4fc4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:37.056022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff208-3224-46d8-9de9-81656e4fc4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:37.056568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff208-3224-46d8-9de9-81656e4fc4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:37.056854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff208-3224-46d8-9de9-81656e4fc4b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:37.058026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ff208-3224-46d8-9de9-81656e4fc4b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:37.062722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ff208-3224-46d8-9de9-81656e4fc4b9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:37.123999 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1114/2232] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:36 2022] POST /identity/v3/projects => generated 387 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:37.125535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac306e4-7505-4b62-9e3b-359be7617c15 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:37.150461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac306e4-7505-4b62-9e3b-359be7617c15 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:37.296445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f862896-7ad3-4159-bd72-ab6b680e0bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:37.297445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f862896-7ad3-4159-bd72-ab6b680e0bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:37.297445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f862896-7ad3-4159-bd72-ab6b680e0bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:37.297617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f862896-7ad3-4159-bd72-ab6b680e0bf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:37.298218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f862896-7ad3-4159-bd72-ab6b680e0bf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:37.300241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f862896-7ad3-4159-bd72-ab6b680e0bf6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:37.369294 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1119/2233] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:37 2022] POST /identity/v3/users => generated 300 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:37.374476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff208-3224-46d8-9de9-81656e4fc4b9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:37.398078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff208-3224-46d8-9de9-81656e4fc4b9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:37.423787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd178d0-ecb1-4b9e-ae07-0c2471f81f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:37.424776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd178d0-ecb1-4b9e-ae07-0c2471f81f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:37.426184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd178d0-ecb1-4b9e-ae07-0c2471f81f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:37.426713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd178d0-ecb1-4b9e-ae07-0c2471f81f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:37.427927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd178d0-ecb1-4b9e-ae07-0c2471f81f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:37.430058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd178d0-ecb1-4b9e-ae07-0c2471f81f2a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:37.486955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1115/2234] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:37.488888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f862896-7ad3-4159-bd72-ab6b680e0bf6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:37.525055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f862896-7ad3-4159-bd72-ab6b680e0bf6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:37.721966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab46c0-3581-4505-a725-517800d85b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:37.724065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab46c0-3581-4505-a725-517800d85b88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:37.725414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab46c0-3581-4505-a725-517800d85b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:37.730675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab46c0-3581-4505-a725-517800d85b88 admin admin] PATH_INFO: `/v3/projects/9ffb7a9491eb46fb8dd0210d1d3e5110/users/35e3f5601474416f9ff9b025d68e8a4d/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:37.816149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab46c0-3581-4505-a725-517800d85b88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffb7a9491eb46fb8dd0210d1d3e5110, user_id=35e3f5601474416f9ff9b025d68e8a4d, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:37.819391 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab46c0-3581-4505-a725-517800d85b88 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:37.871048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1120/2235] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:37 2022] PUT /identity/v3/projects/9ffb7a9491eb46fb8dd0210d1d3e5110/users/35e3f5601474416f9ff9b025d68e8a4d/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:37.875223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd178d0-ecb1-4b9e-ae07-0c2471f81f2a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:37.892311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd178d0-ecb1-4b9e-ae07-0c2471f81f2a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:38.035588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db0256-9303-4d72-8e55-24f77ccc7841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:38.036241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db0256-9303-4d72-8e55-24f77ccc7841 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:38.036631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db0256-9303-4d72-8e55-24f77ccc7841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:38.036910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db0256-9303-4d72-8e55-24f77ccc7841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:38.037726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db0256-9303-4d72-8e55-24f77ccc7841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:38.039388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db0256-9303-4d72-8e55-24f77ccc7841 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:38.066575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1116/2236] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:38.066987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab46c0-3581-4505-a725-517800d85b88 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:38.086526 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab46c0-3581-4505-a725-517800d85b88 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:38.102842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b014998-3677-41f9-9023-d3760a00908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:38.104037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014998-3677-41f9-9023-d3760a00908c admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:38.104547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014998-3677-41f9-9023-d3760a00908c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:38.104992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014998-3677-41f9-9023-d3760a00908c admin admin] PATH_INFO: `/v3/projects/9ffb7a9491eb46fb8dd0210d1d3e5110/users/35e3f5601474416f9ff9b025d68e8a4d/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:38.110491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014998-3677-41f9-9023-d3760a00908c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffb7a9491eb46fb8dd0210d1d3e5110, user_id=35e3f5601474416f9ff9b025d68e8a4d, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:38.114050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014998-3677-41f9-9023-d3760a00908c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:38.136486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1121/2237] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:38 2022] PUT /identity/v3/projects/9ffb7a9491eb46fb8dd0210d1d3e5110/users/35e3f5601474416f9ff9b025d68e8a4d/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:25:38.146129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ea1dc-809b-4254-96d3-4df1873a27fb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:38.146418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ea1dc-809b-4254-96d3-4df1873a27fb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:38.146702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ea1dc-809b-4254-96d3-4df1873a27fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:38.207898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-eb6ea1dc-809b-4254-96d3-4df1873a27fb None None] MFA Rules not processed for user `35e3f5601474416f9ff9b025d68e8a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:38.209276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-eb6ea1dc-809b-4254-96d3-4df1873a27fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:38.239258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:38.411132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1117/2238] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:38.419033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b014998-3677-41f9-9023-d3760a00908c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:38.422379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b014998-3677-41f9-9023-d3760a00908c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:38.599918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b014998-3677-41f9-9023-d3760a00908c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:38.622191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61a873-85fb-4d22-bef4-fb39f1d22ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:38.622899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61a873-85fb-4d22-bef4-fb39f1d22ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:38.623297 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61a873-85fb-4d22-bef4-fb39f1d22ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:38.623576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61a873-85fb-4d22-bef4-fb39f1d22ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:38.644131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61a873-85fb-4d22-bef4-fb39f1d22ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:38.648585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61a873-85fb-4d22-bef4-fb39f1d22ff1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:38.662005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1122/2239] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:38.765143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ea1dc-809b-4254-96d3-4df1873a27fb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:38.892671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52215d76-9b2e-4632-8833-4813a1ca695a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:38.893712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52215d76-9b2e-4632-8833-4813a1ca695a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:38.894176 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52215d76-9b2e-4632-8833-4813a1ca695a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:38.934247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-52215d76-9b2e-4632-8833-4813a1ca695a None None] MFA Rules not processed for user `4aaa94a183fb49b79e6383ea783ab76d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:38.938301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-52215d76-9b2e-4632-8833-4813a1ca695a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:38.959049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:39.071544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ea1dc-809b-4254-96d3-4df1873a27fb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:39.083339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:39.084280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:39.084766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:39.085207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] PATH_INFO: `/v3/users/5d61454f9f464e22938a1ab9a9c603ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:39.086708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d61454f9f464e22938a1ab9a9c603ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:39.089462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:39.091384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1123/2240] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:39.247597 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d61454f9f464e22938a1ab9a9c603ae'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:39.298589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d61454f9f464e22938a1ab9a9c603ae'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:39.299043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d61454f9f464e22938a1ab9a9c603ae'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:39.332670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d61454f9f464e22938a1ab9a9c603ae'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:39.348652 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d61454f9f464e22938a1ab9a9c603ae'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:39.364443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1118/2241] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:38 2022] DELETE /identity/v3/users/5d61454f9f464e22938a1ab9a9c603ae => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:39.367129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc6c4c-1f9d-45c1-9fc2-030dd9af36bc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:39.367744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc6c4c-1f9d-45c1-9fc2-030dd9af36bc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:39.368231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc6c4c-1f9d-45c1-9fc2-030dd9af36bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:39.424572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b5dc6c4c-1f9d-45c1-9fc2-030dd9af36bc None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:39.425833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b5dc6c4c-1f9d-45c1-9fc2-030dd9af36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:39.655822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1124/2242] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:39.723780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:39.725479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:39.816078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:39.873932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a38e8f-ca8e-4c1f-a30b-db68d9dea7d9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:39.886206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f38de-8b3f-424f-b996-58b07aacd3be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:39.888846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f38de-8b3f-424f-b996-58b07aacd3be service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:39.889384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f38de-8b3f-424f-b996-58b07aacd3be service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:39.890117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f38de-8b3f-424f-b996-58b07aacd3be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:39.932427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f38de-8b3f-424f-b996-58b07aacd3be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:39.937923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f38de-8b3f-424f-b996-58b07aacd3be service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:39.970005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1119/2243] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:41.346239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc6c4c-1f9d-45c1-9fc2-030dd9af36bc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:41.350341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b5dc6c4c-1f9d-45c1-9fc2-030dd9af36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:41.381664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc6c4c-1f9d-45c1-9fc2-030dd9af36bc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:41.437775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:41.438996 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:41.440664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:41.450630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] PATH_INFO: `/v3/projects/06d64e6f093249f99bb85493be3ca1a5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:41.456058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d64e6f093249f99bb85493be3ca1a5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:41.478714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:41.721495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d64e6f093249f99bb85493be3ca1a5', 'request_id': 'req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:41.746826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d64e6f093249f99bb85493be3ca1a5', 'request_id': 'req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:41.785146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d64e6f093249f99bb85493be3ca1a5', 'request_id': 'req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:41.821620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] The token cache is being invalidate because project 06d64e6f093249f99bb85493be3ca1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:41.821904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:41.822935 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9ff1a19b-6f9a-46a3-899d-aaf0f82b1aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:41.829046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1125/2244] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:41 2022] DELETE /identity/v3/projects/06d64e6f093249f99bb85493be3ca1a5 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:41.834864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f38de-8b3f-424f-b996-58b07aacd3be service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:41.884772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f38de-8b3f-424f-b996-58b07aacd3be service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:42.213406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:42.214802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:42.220315 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:42.221603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] PATH_INFO: `/v3/users/35e3f5601474416f9ff9b025d68e8a4d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:42.225395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e3f5601474416f9ff9b025d68e8a4d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:42.229380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:42.255480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75ad30-d2f1-447f-bd91-08b18e7f80e2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:42.256029 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75ad30-d2f1-447f-bd91-08b18e7f80e2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:42.256368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75ad30-d2f1-447f-bd91-08b18e7f80e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:42.361342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1d75ad30-d2f1-447f-bd91-08b18e7f80e2 None None] MFA Rules not processed for user `af3e7eea0b2840b79404b370382ab44a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:42.363724 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1d75ad30-d2f1-447f-bd91-08b18e7f80e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:42.440681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e3f5601474416f9ff9b025d68e8a4d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:42.452029 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:42.466470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e3f5601474416f9ff9b025d68e8a4d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:42.467467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e3f5601474416f9ff9b025d68e8a4d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:42.504286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e3f5601474416f9ff9b025d68e8a4d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:42.525615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e3f5601474416f9ff9b025d68e8a4d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:42.534227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1120/2245] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:41 2022] DELETE /identity/v3/users/35e3f5601474416f9ff9b025d68e8a4d => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:42.592686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1126/2246] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:42.596529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:42.599199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:42.798609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:42.875183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7480d-a333-4196-9517-49ca1c33c12b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:42.888714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a4ab1-4b56-467e-963c-d8fa1f2e6573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:42.889930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a4ab1-4b56-467e-963c-d8fa1f2e6573 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:42.890314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a4ab1-4b56-467e-963c-d8fa1f2e6573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:42.890862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a4ab1-4b56-467e-963c-d8fa1f2e6573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:42.891636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a4ab1-4b56-467e-963c-d8fa1f2e6573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:42.895622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a4ab1-4b56-467e-963c-d8fa1f2e6573 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:42.994080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1121/2247] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:42 2022] POST /identity/v3/projects => generated 379 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:42.998034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75ad30-d2f1-447f-bd91-08b18e7f80e2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:43.014966 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75ad30-d2f1-447f-bd91-08b18e7f80e2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:43.246291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb1cb5-4e87-4e9e-a969-2db83625ce3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:43.247049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb1cb5-4e87-4e9e-a969-2db83625ce3b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:43.247337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb1cb5-4e87-4e9e-a969-2db83625ce3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:43.247611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb1cb5-4e87-4e9e-a969-2db83625ce3b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:43.248269 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb1cb5-4e87-4e9e-a969-2db83625ce3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:43.250350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb1cb5-4e87-4e9e-a969-2db83625ce3b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:43.359971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a4ab1-4b56-467e-963c-d8fa1f2e6573 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:43.363861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-158a4ab1-4b56-467e-963c-d8fa1f2e6573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:43.365039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1127/2248] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:42 2022] POST /identity/v3/users => generated 296 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:43.370846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb1cb5-4e87-4e9e-a969-2db83625ce3b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:43.388404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb1cb5-4e87-4e9e-a969-2db83625ce3b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:43.403398 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ec527-3d4a-4f6b-802f-d5071c8d12cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:43.404242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ec527-3d4a-4f6b-802f-d5071c8d12cc admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:43.404657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ec527-3d4a-4f6b-802f-d5071c8d12cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:43.405014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ec527-3d4a-4f6b-802f-d5071c8d12cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:43.406340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ec527-3d4a-4f6b-802f-d5071c8d12cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:43.409911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ec527-3d4a-4f6b-802f-d5071c8d12cc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:43.415631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a4ab1-4b56-467e-963c-d8fa1f2e6573 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:43.448912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1128/2249] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:43.454088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ec527-3d4a-4f6b-802f-d5071c8d12cc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:43.456673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:43.457308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:43.457588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:43.457863 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] PATH_INFO: `/v3/projects/9ffb7a9491eb46fb8dd0210d1d3e5110` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:43.463301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ffb7a9491eb46fb8dd0210d1d3e5110)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:43.465544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:43.478476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ec527-3d4a-4f6b-802f-d5071c8d12cc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:43.497222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a5aed-d824-4718-b006-cb53c23214cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:43.500109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a5aed-d824-4718-b006-cb53c23214cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:43.500775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a5aed-d824-4718-b006-cb53c23214cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:43.502470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a5aed-d824-4718-b006-cb53c23214cd admin admin] PATH_INFO: `/v3/projects/7c817ad50ec0407b81e579cfd37a47a4/users/12e33953d0414c13ab92a40c7cddb7b8/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:43.581711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a5aed-d824-4718-b006-cb53c23214cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c817ad50ec0407b81e579cfd37a47a4, user_id=12e33953d0414c13ab92a40c7cddb7b8, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:43.611445 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a5aed-d824-4718-b006-cb53c23214cd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:43.622304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ffb7a9491eb46fb8dd0210d1d3e5110', 'request_id': 'req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:43.644044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ffb7a9491eb46fb8dd0210d1d3e5110', 'request_id': 'req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:43.662059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1129/2250] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:43 2022] PUT /identity/v3/projects/7c817ad50ec0407b81e579cfd37a47a4/users/12e33953d0414c13ab92a40c7cddb7b8/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:43.672537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a5aed-d824-4718-b006-cb53c23214cd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:43.681936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ffb7a9491eb46fb8dd0210d1d3e5110', 'request_id': 'req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:43.758998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] The token cache is being invalidate because project 9ffb7a9491eb46fb8dd0210d1d3e5110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:43.759620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:43.761666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f93d811-3bc9-4e25-89f1-4bfe7b919ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:43.767347 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1122/2251] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:43 2022] DELETE /identity/v3/projects/9ffb7a9491eb46fb8dd0210d1d3e5110 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:43.769151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165bf0d-ab8b-439e-be52-3e37bea068c4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:43.769780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165bf0d-ab8b-439e-be52-3e37bea068c4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:43.770758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165bf0d-ab8b-439e-be52-3e37bea068c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:43.820813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6165bf0d-ab8b-439e-be52-3e37bea068c4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:43.833062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6165bf0d-ab8b-439e-be52-3e37bea068c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:43.991688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a5aed-d824-4718-b006-cb53c23214cd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:44.005045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118eca4-98bb-4d1c-be51-d25ea83d829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:44.008838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118eca4-98bb-4d1c-be51-d25ea83d829d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.008838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118eca4-98bb-4d1c-be51-d25ea83d829d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.008838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118eca4-98bb-4d1c-be51-d25ea83d829d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:44.008838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118eca4-98bb-4d1c-be51-d25ea83d829d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:44.014621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118eca4-98bb-4d1c-be51-d25ea83d829d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:44.040761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1130/2252] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:44.046110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1123/2253] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:44.047884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118eca4-98bb-4d1c-be51-d25ea83d829d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:44.053254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0118eca4-98bb-4d1c-be51-d25ea83d829d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:44.055740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09db96f-c4fe-4494-9620-9ef95fd12848 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.056274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09db96f-c4fe-4494-9620-9ef95fd12848 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.056812 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09db96f-c4fe-4494-9620-9ef95fd12848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:44.090001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a09db96f-c4fe-4494-9620-9ef95fd12848 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:44.106906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a09db96f-c4fe-4494-9620-9ef95fd12848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:44.110960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118eca4-98bb-4d1c-be51-d25ea83d829d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:44.121809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1124/2254] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:44.128127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09db96f-c4fe-4494-9620-9ef95fd12848 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:44.141587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09db96f-c4fe-4494-9620-9ef95fd12848 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:44.143460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916d8b5-c857-4d30-9cea-4976867a4e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:44.145880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916d8b5-c857-4d30-9cea-4976867a4e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.146172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916d8b5-c857-4d30-9cea-4976867a4e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.146449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916d8b5-c857-4d30-9cea-4976867a4e19 admin admin] PATH_INFO: `/v3/projects/7c817ad50ec0407b81e579cfd37a47a4/users/12e33953d0414c13ab92a40c7cddb7b8/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:44.149935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd793a03-4be4-42ca-9898-91e991ed4d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:44.150876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd793a03-4be4-42ca-9898-91e991ed4d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.151661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd793a03-4be4-42ca-9898-91e991ed4d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.152244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd793a03-4be4-42ca-9898-91e991ed4d39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:44.153201 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd793a03-4be4-42ca-9898-91e991ed4d39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:25:44.153757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd793a03-4be4-42ca-9898-91e991ed4d39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:44.156160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd793a03-4be4-42ca-9898-91e991ed4d39 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:44.157679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916d8b5-c857-4d30-9cea-4976867a4e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c817ad50ec0407b81e579cfd37a47a4, user_id=12e33953d0414c13ab92a40c7cddb7b8, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:44.171342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916d8b5-c857-4d30-9cea-4976867a4e19 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:44.174695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1125/2255] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:25:44.179775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd793a03-4be4-42ca-9898-91e991ed4d39 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:44.205394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd793a03-4be4-42ca-9898-91e991ed4d39 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:44.214864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1131/2256] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:44 2022] PUT /identity/v3/projects/7c817ad50ec0407b81e579cfd37a47a4/users/12e33953d0414c13ab92a40c7cddb7b8/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:44.299775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:44.341706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f163d14-d5e6-411f-b4e9-371b8e286338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:44.342384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f163d14-d5e6-411f-b4e9-371b8e286338 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.342669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f163d14-d5e6-411f-b4e9-371b8e286338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.342941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f163d14-d5e6-411f-b4e9-371b8e286338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:44.345056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f163d14-d5e6-411f-b4e9-371b8e286338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:44.351719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f163d14-d5e6-411f-b4e9-371b8e286338 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:44.438907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1126/2257] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:44 2022] POST /identity/v3/projects => generated 407 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:44.443327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916d8b5-c857-4d30-9cea-4976867a4e19 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:44.474977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916d8b5-c857-4d30-9cea-4976867a4e19 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:44.484233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:44.569943 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270096c-9afc-44fc-a6fc-a043248d54db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:44.570593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270096c-9afc-44fc-a6fc-a043248d54db admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.570871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270096c-9afc-44fc-a6fc-a043248d54db admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.571141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270096c-9afc-44fc-a6fc-a043248d54db admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:44.571977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270096c-9afc-44fc-a6fc-a043248d54db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:44.575304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270096c-9afc-44fc-a6fc-a043248d54db admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:44.639128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f163d14-d5e6-411f-b4e9-371b8e286338 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:44.644652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1132/2258] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:44 2022] POST /identity/v3/users => generated 304 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:44.659049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f163d14-d5e6-411f-b4e9-371b8e286338 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:44.671615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d06600-149a-4520-9ee0-994ae3d8aab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:44.673956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d06600-149a-4520-9ee0-994ae3d8aab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.689773 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d06600-149a-4520-9ee0-994ae3d8aab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.689913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d06600-149a-4520-9ee0-994ae3d8aab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:44.690587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d06600-149a-4520-9ee0-994ae3d8aab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:44.692409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d06600-149a-4520-9ee0-994ae3d8aab8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:44.708760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1127/2259] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:44.710162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270096c-9afc-44fc-a6fc-a043248d54db admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:44.725940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270096c-9afc-44fc-a6fc-a043248d54db admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:44.738466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb5587-c801-4102-9fbe-53b547c6d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:44.739109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb5587-c801-4102-9fbe-53b547c6d2ce admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.739392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb5587-c801-4102-9fbe-53b547c6d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.739669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb5587-c801-4102-9fbe-53b547c6d2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:44.741577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb5587-c801-4102-9fbe-53b547c6d2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:44.746375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb5587-c801-4102-9fbe-53b547c6d2ce admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:44.774965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1133/2260] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:44.780223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d06600-149a-4520-9ee0-994ae3d8aab8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:44.806384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d06600-149a-4520-9ee0-994ae3d8aab8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:44.827245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24f1a6-41c0-4b5c-9d64-4b785bdc6ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:44.829813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24f1a6-41c0-4b5c-9d64-4b785bdc6ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.829813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24f1a6-41c0-4b5c-9d64-4b785bdc6ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.831136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24f1a6-41c0-4b5c-9d64-4b785bdc6ff6 admin admin] PATH_INFO: `/v3/projects/4789bbc860074033b51b7e4c6c7334b7/users/6c8768fe8d2543c2964db24ed3854342/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:44.919364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24f1a6-41c0-4b5c-9d64-4b785bdc6ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4789bbc860074033b51b7e4c6c7334b7, user_id=6c8768fe8d2543c2964db24ed3854342, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:44.927185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24f1a6-41c0-4b5c-9d64-4b785bdc6ff6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:44.966411 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1128/2261] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:44 2022] PUT /identity/v3/projects/4789bbc860074033b51b7e4c6c7334b7/users/6c8768fe8d2543c2964db24ed3854342/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:44.986990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f4498-067e-458b-8044-7fcc4741e5cd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:44.987282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f4498-067e-458b-8044-7fcc4741e5cd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:44.987764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f4498-067e-458b-8044-7fcc4741e5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:45.054411 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-532f4498-067e-458b-8044-7fcc4741e5cd None None] MFA Rules not processed for user `6c8768fe8d2543c2964db24ed3854342`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:45.055679 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-532f4498-067e-458b-8044-7fcc4741e5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:45.348532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24f1a6-41c0-4b5c-9d64-4b785bdc6ff6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:45.364371 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1134/2262] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:44 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:45.544431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24f1a6-41c0-4b5c-9d64-4b785bdc6ff6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:45.558280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475d3aa-9096-4b0f-b73c-945b6d06e6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:45.558824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475d3aa-9096-4b0f-b73c-945b6d06e6ae admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:45.559192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475d3aa-9096-4b0f-b73c-945b6d06e6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:45.559537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475d3aa-9096-4b0f-b73c-945b6d06e6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:45.560923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475d3aa-9096-4b0f-b73c-945b6d06e6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:45.564023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475d3aa-9096-4b0f-b73c-945b6d06e6ae admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:45.647772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1129/2263] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:25:45 2022] POST /identity/v3/projects => generated 407 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:45.649587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f4498-067e-458b-8044-7fcc4741e5cd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:45.678844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f4498-067e-458b-8044-7fcc4741e5cd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:45.944473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719a537-a698-4419-838c-9eaf9ab2505b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:45.945105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719a537-a698-4419-838c-9eaf9ab2505b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:45.945367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719a537-a698-4419-838c-9eaf9ab2505b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:45.945644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719a537-a698-4419-838c-9eaf9ab2505b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:45.946685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719a537-a698-4419-838c-9eaf9ab2505b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:45.951866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719a537-a698-4419-838c-9eaf9ab2505b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:46.007925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1135/2264] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:25:45 2022] POST /identity/v3/users => generated 310 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:25:46.008995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475d3aa-9096-4b0f-b73c-945b6d06e6ae admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:46.031730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475d3aa-9096-4b0f-b73c-945b6d06e6ae admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:46.045661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f2050-a584-4fe0-b45d-74a6f6d26a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:46.048914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f2050-a584-4fe0-b45d-74a6f6d26a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:46.049470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f2050-a584-4fe0-b45d-74a6f6d26a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:46.049937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f2050-a584-4fe0-b45d-74a6f6d26a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:46.052582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f2050-a584-4fe0-b45d-74a6f6d26a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:46.057243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f2050-a584-4fe0-b45d-74a6f6d26a8a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:46.086222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1130/2265] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:25:46.091114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719a537-a698-4419-838c-9eaf9ab2505b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:46.110146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:46.117658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719a537-a698-4419-838c-9eaf9ab2505b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:46.133736 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a3961-9c95-44b4-ade2-07028272369b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:46.139472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a3961-9c95-44b4-ade2-07028272369b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:46.140141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a3961-9c95-44b4-ade2-07028272369b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:46.142525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a3961-9c95-44b4-ade2-07028272369b admin admin] PATH_INFO: `/v3/projects/a8e917021c124dfaa2849f8058062687/users/4b27bfca1a3d45b6ab7e678866460f6b/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:46.200666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a3961-9c95-44b4-ade2-07028272369b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e917021c124dfaa2849f8058062687, user_id=4b27bfca1a3d45b6ab7e678866460f6b, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:46.203580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a3961-9c95-44b4-ade2-07028272369b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:46.237707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1136/2266] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:46 2022] PUT /identity/v3/projects/a8e917021c124dfaa2849f8058062687/users/4b27bfca1a3d45b6ab7e678866460f6b/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:46.241543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f2050-a584-4fe0-b45d-74a6f6d26a8a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:46.259402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f2050-a584-4fe0-b45d-74a6f6d26a8a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:46.366480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750ba17-73d1-4c13-b8d2-ac9e028c14ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:46.369063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750ba17-73d1-4c13-b8d2-ac9e028c14ad admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:46.369560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750ba17-73d1-4c13-b8d2-ac9e028c14ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:46.371274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750ba17-73d1-4c13-b8d2-ac9e028c14ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:46.372676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750ba17-73d1-4c13-b8d2-ac9e028c14ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:46.374642 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750ba17-73d1-4c13-b8d2-ac9e028c14ad admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:46.389017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1131/2267] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:25:46.392439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a3961-9c95-44b4-ade2-07028272369b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:46.425350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a3961-9c95-44b4-ade2-07028272369b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:46.434205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92487eaa-643d-45d8-92c8-41ddd4087d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:46.437100 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92487eaa-643d-45d8-92c8-41ddd4087d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:46.440062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92487eaa-643d-45d8-92c8-41ddd4087d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:46.442256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92487eaa-643d-45d8-92c8-41ddd4087d03 admin admin] PATH_INFO: `/v3/projects/a8e917021c124dfaa2849f8058062687/users/4b27bfca1a3d45b6ab7e678866460f6b/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:46.451209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92487eaa-643d-45d8-92c8-41ddd4087d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e917021c124dfaa2849f8058062687, user_id=4b27bfca1a3d45b6ab7e678866460f6b, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:46.454494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92487eaa-643d-45d8-92c8-41ddd4087d03 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:46.508145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1137/2268] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:25:46 2022] PUT /identity/v3/projects/a8e917021c124dfaa2849f8058062687/users/4b27bfca1a3d45b6ab7e678866460f6b/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:46.519727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bdc00-9aaf-456b-8c5c-317b7b000569 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:46.520800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bdc00-9aaf-456b-8c5c-317b7b000569 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:46.521582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bdc00-9aaf-456b-8c5c-317b7b000569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:46.534051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:46.596453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-141bdc00-9aaf-456b-8c5c-317b7b000569 None None] MFA Rules not processed for user `4b27bfca1a3d45b6ab7e678866460f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:46.596453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-141bdc00-9aaf-456b-8c5c-317b7b000569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:46.834357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1132/2269] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:46 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:46.840273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92487eaa-643d-45d8-92c8-41ddd4087d03 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:46.842768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-92487eaa-643d-45d8-92c8-41ddd4087d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:46.962238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92487eaa-643d-45d8-92c8-41ddd4087d03 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:46.971047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479f171-c797-4e88-891f-b7b78607f0d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:46.971684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479f171-c797-4e88-891f-b7b78607f0d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:46.971965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479f171-c797-4e88-891f-b7b78607f0d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:46.972239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479f171-c797-4e88-891f-b7b78607f0d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:46.993439 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479f171-c797-4e88-891f-b7b78607f0d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:46.996309 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479f171-c797-4e88-891f-b7b78607f0d0 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:47.012343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1138/2270] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:46 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:47.357822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bdc00-9aaf-456b-8c5c-317b7b000569 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:47.377696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bdc00-9aaf-456b-8c5c-317b7b000569 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:47.384259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92977a01-03a5-4c99-8127-c999556e6363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:47.384750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92977a01-03a5-4c99-8127-c999556e6363 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:47.385130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92977a01-03a5-4c99-8127-c999556e6363 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:47.385585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92977a01-03a5-4c99-8127-c999556e6363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:47.504596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92977a01-03a5-4c99-8127-c999556e6363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:47.506303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92977a01-03a5-4c99-8127-c999556e6363 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:47.519676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1133/2271] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:47 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:57.002860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=39,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:57.185389 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33775dab-4ca0-4045-8a24-4f57ee510285 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:57.185668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33775dab-4ca0-4045-8a24-4f57ee510285 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:57.185933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33775dab-4ca0-4045-8a24-4f57ee510285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:57.228229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-33775dab-4ca0-4045-8a24-4f57ee510285 None None] MFA Rules not processed for user `12e33953d0414c13ab92a40c7cddb7b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:57.230370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-33775dab-4ca0-4045-8a24-4f57ee510285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:57.341906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1139/2272] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:25:57.357223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92977a01-03a5-4c99-8127-c999556e6363 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:57.359358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-92977a01-03a5-4c99-8127-c999556e6363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:57.486815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92977a01-03a5-4c99-8127-c999556e6363 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:57.496176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5f69d-69bc-49eb-ae06-352ffa42ca71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:57.497997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f69d-69bc-49eb-ae06-352ffa42ca71 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:57.498303 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f69d-69bc-49eb-ae06-352ffa42ca71 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:57.498578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f69d-69bc-49eb-ae06-352ffa42ca71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:57.500668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8ae5f69d-69bc-49eb-ae06-352ffa42ca71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:57.559108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:57.624094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33775dab-4ca0-4045-8a24-4f57ee510285 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:57.656136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5f69d-69bc-49eb-ae06-352ffa42ca71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:57.662728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5f69d-69bc-49eb-ae06-352ffa42ca71 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:57.762388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1134/2273] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:57.827272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33775dab-4ca0-4045-8a24-4f57ee510285 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:57.836716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:57.837452 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:57.837843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:57.838220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] PATH_INFO: `/v3/users/6c8768fe8d2543c2964db24ed3854342` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:57.841836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8768fe8d2543c2964db24ed3854342)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:57.845734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:57.945397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8768fe8d2543c2964db24ed3854342'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:57.946934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8768fe8d2543c2964db24ed3854342'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:57.947728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8768fe8d2543c2964db24ed3854342'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:57.975092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8768fe8d2543c2964db24ed3854342'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:58.014378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8768fe8d2543c2964db24ed3854342'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:58.039713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1140/2274] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:57 2022] DELETE /identity/v3/users/6c8768fe8d2543c2964db24ed3854342 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:58.044971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dd93b-8c4f-4815-a12d-c7dd789f8478 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:58.045513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dd93b-8c4f-4815-a12d-c7dd789f8478 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:58.045951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dd93b-8c4f-4815-a12d-c7dd789f8478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:58.071948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e67dd93b-8c4f-4815-a12d-c7dd789f8478 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:25:58.074277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e67dd93b-8c4f-4815-a12d-c7dd789f8478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:58.208377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1135/2275] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:25:58.222663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:58.224377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:58.234236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:25:58.371941 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e68ec4a-3c7b-49b4-9519-eb4857861f0d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:58.376554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153882b-55e1-49d1-b157-042e9bc47718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:58.377563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153882b-55e1-49d1-b157-042e9bc47718 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:58.378674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153882b-55e1-49d1-b157-042e9bc47718 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:58.379273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153882b-55e1-49d1-b157-042e9bc47718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:58.404106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153882b-55e1-49d1-b157-042e9bc47718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:58.406659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153882b-55e1-49d1-b157-042e9bc47718 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:58.457710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1141/2276] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:25:59.050142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dd93b-8c4f-4815-a12d-c7dd789f8478 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:59.051964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e67dd93b-8c4f-4815-a12d-c7dd789f8478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:25:59.101169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dd93b-8c4f-4815-a12d-c7dd789f8478 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:59.126914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:59.127571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:59.127849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:59.128120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] PATH_INFO: `/v3/projects/4789bbc860074033b51b7e4c6c7334b7` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:59.130047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4789bbc860074033b51b7e4c6c7334b7)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:59.132042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:59.256971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4789bbc860074033b51b7e4c6c7334b7', 'request_id': 'req-8b050bbd-7c59-443f-aec9-1e6ebf318c75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.279220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4789bbc860074033b51b7e4c6c7334b7', 'request_id': 'req-8b050bbd-7c59-443f-aec9-1e6ebf318c75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.297291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4789bbc860074033b51b7e4c6c7334b7', 'request_id': 'req-8b050bbd-7c59-443f-aec9-1e6ebf318c75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.323164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] The token cache is being invalidate because project 4789bbc860074033b51b7e4c6c7334b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:25:59.323855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.324819 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.341378 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1136/2277] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:25:59 2022] DELETE /identity/v3/projects/4789bbc860074033b51b7e4c6c7334b7 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:25:59.346067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153882b-55e1-49d1-b157-042e9bc47718 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:25:59.383577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153882b-55e1-49d1-b157-042e9bc47718 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:25:59.663275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:25:59.664322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:25:59.664847 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:25:59.665351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] PATH_INFO: `/v3/users/4b27bfca1a3d45b6ab7e678866460f6b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:25:59.667787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b27bfca1a3d45b6ab7e678866460f6b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:25:59.670612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:25:59.796471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b27bfca1a3d45b6ab7e678866460f6b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.798786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b27bfca1a3d45b6ab7e678866460f6b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.800893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b27bfca1a3d45b6ab7e678866460f6b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.837487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b27bfca1a3d45b6ab7e678866460f6b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.872560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1f81ed5c-485e-4b00-b288-2082003c7c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b27bfca1a3d45b6ab7e678866460f6b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:25:59.923754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1142/2278] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:25:59 2022] DELETE /identity/v3/users/4b27bfca1a3d45b6ab7e678866460f6b => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:01.063111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:01.064836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:01.093611 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b050bbd-7c59-443f-aec9-1e6ebf318c75 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:01.223506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:01.266450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:01.269547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:01.270007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:01.271004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] PATH_INFO: `/v3/projects/a8e917021c124dfaa2849f8058062687` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:01.281297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8e917021c124dfaa2849f8058062687)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:01.284107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:01.454654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8e917021c124dfaa2849f8058062687', 'request_id': 'req-d526dcf5-de7e-40f1-9a76-5aec67f51d59'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:01.489663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8e917021c124dfaa2849f8058062687', 'request_id': 'req-d526dcf5-de7e-40f1-9a76-5aec67f51d59'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:01.512765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8e917021c124dfaa2849f8058062687', 'request_id': 'req-d526dcf5-de7e-40f1-9a76-5aec67f51d59'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:01.532561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] The token cache is being invalidate because project a8e917021c124dfaa2849f8058062687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:26:01.533166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:01.534745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d526dcf5-de7e-40f1-9a76-5aec67f51d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:01.540104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1137/2279] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:26:01 2022] DELETE /identity/v3/projects/a8e917021c124dfaa2849f8058062687 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:01.545510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9034bb3-67ab-456c-a89e-a19e6ed78e85 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:01.546181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9034bb3-67ab-456c-a89e-a19e6ed78e85 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:01.546181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9034bb3-67ab-456c-a89e-a19e6ed78e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:01.581472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e9034bb3-67ab-456c-a89e-a19e6ed78e85 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:01.582780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e9034bb3-67ab-456c-a89e-a19e6ed78e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:01.656593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:01.744715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1143/2280] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:01.748225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbc669-13d2-499e-a9fc-1d1a0eda2821 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:01.749136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbc669-13d2-499e-a9fc-1d1a0eda2821 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:01.749562 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbc669-13d2-499e-a9fc-1d1a0eda2821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:01.798505 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5cfbc669-13d2-499e-a9fc-1d1a0eda2821 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:01.802723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5cfbc669-13d2-499e-a9fc-1d1a0eda2821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:01.823911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1138/2281] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:01.825349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9034bb3-67ab-456c-a89e-a19e6ed78e85 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:01.839848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9034bb3-67ab-456c-a89e-a19e6ed78e85 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:01.860612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959ba01-e2fc-40fc-bae5-a8a93330df7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:01.872222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959ba01-e2fc-40fc-bae5-a8a93330df7b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:01.872546 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959ba01-e2fc-40fc-bae5-a8a93330df7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:01.872830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959ba01-e2fc-40fc-bae5-a8a93330df7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:01.873578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959ba01-e2fc-40fc-bae5-a8a93330df7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:26:01.877886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959ba01-e2fc-40fc-bae5-a8a93330df7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:01.881644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959ba01-e2fc-40fc-bae5-a8a93330df7b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:01.921880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1144/2282] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:26:01.923096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbc669-13d2-499e-a9fc-1d1a0eda2821 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:01.942804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbc669-13d2-499e-a9fc-1d1a0eda2821 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:01.956520 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd6f0c-a0d5-44ca-a334-fd7874aa884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:01.957147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd6f0c-a0d5-44ca-a334-fd7874aa884c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:01.957428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd6f0c-a0d5-44ca-a334-fd7874aa884c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:01.957702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd6f0c-a0d5-44ca-a334-fd7874aa884c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:01.958343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd6f0c-a0d5-44ca-a334-fd7874aa884c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:01.960368 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd6f0c-a0d5-44ca-a334-fd7874aa884c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.034379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959ba01-e2fc-40fc-bae5-a8a93330df7b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:02.034523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1139/2283] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:26:01 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:02.049199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959ba01-e2fc-40fc-bae5-a8a93330df7b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:02.161586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb5355-a7d9-4fd3-a42a-946c097a92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:02.162869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb5355-a7d9-4fd3-a42a-946c097a92ae admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.163004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb5355-a7d9-4fd3-a42a-946c097a92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.163326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb5355-a7d9-4fd3-a42a-946c097a92ae admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.164053 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb5355-a7d9-4fd3-a42a-946c097a92ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:02.166212 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb5355-a7d9-4fd3-a42a-946c097a92ae admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.215098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1145/2284] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:26:02 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:02.216828 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd6f0c-a0d5-44ca-a334-fd7874aa884c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:02.235538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd6f0c-a0d5-44ca-a334-fd7874aa884c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:02.249987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391dec2-ba1c-4a4d-a63c-53ec7e5705fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:02.250599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391dec2-ba1c-4a4d-a63c-53ec7e5705fb admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.250881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391dec2-ba1c-4a4d-a63c-53ec7e5705fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.251893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391dec2-ba1c-4a4d-a63c-53ec7e5705fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.253221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391dec2-ba1c-4a4d-a63c-53ec7e5705fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:02.255766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391dec2-ba1c-4a4d-a63c-53ec7e5705fb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.269895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1140/2285] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:02.275119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb5355-a7d9-4fd3-a42a-946c097a92ae admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:02.289238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb5355-a7d9-4fd3-a42a-946c097a92ae admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:02.326622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc64f34-92ed-46b5-8a68-473c8c4fcc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:02.327253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc64f34-92ed-46b5-8a68-473c8c4fcc5e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.327624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc64f34-92ed-46b5-8a68-473c8c4fcc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.327818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc64f34-92ed-46b5-8a68-473c8c4fcc5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.328759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc64f34-92ed-46b5-8a68-473c8c4fcc5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:02.330241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc64f34-92ed-46b5-8a68-473c8c4fcc5e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.360547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1146/2286] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:02.364297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391dec2-ba1c-4a4d-a63c-53ec7e5705fb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:02.380559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391dec2-ba1c-4a4d-a63c-53ec7e5705fb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:02.386089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbbaff-6df6-4b32-93a5-bd6c94ca2281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:02.386860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbbaff-6df6-4b32-93a5-bd6c94ca2281 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.387137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbbaff-6df6-4b32-93a5-bd6c94ca2281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.387410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbbaff-6df6-4b32-93a5-bd6c94ca2281 admin admin] PATH_INFO: `/v3/projects/b52ac5475a7643329ecfc06cca6e8f73/users/f4bb09d60d244901a236662d6e443282/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.405953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbbaff-6df6-4b32-93a5-bd6c94ca2281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52ac5475a7643329ecfc06cca6e8f73, user_id=f4bb09d60d244901a236662d6e443282, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:02.408714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbbaff-6df6-4b32-93a5-bd6c94ca2281 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.431103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1141/2287] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:02 2022] PUT /identity/v3/projects/b52ac5475a7643329ecfc06cca6e8f73/users/f4bb09d60d244901a236662d6e443282/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:02.442672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013df9ca-06f0-4e10-bd30-cb467ecb67ce None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.443118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013df9ca-06f0-4e10-bd30-cb467ecb67ce None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.443412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013df9ca-06f0-4e10-bd30-cb467ecb67ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.478709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-013df9ca-06f0-4e10-bd30-cb467ecb67ce None None] MFA Rules not processed for user `f4bb09d60d244901a236662d6e443282`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:02.481107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-013df9ca-06f0-4e10-bd30-cb467ecb67ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:02.569815 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1147/2288] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:02.575687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbbaff-6df6-4b32-93a5-bd6c94ca2281 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:02.651918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbbaff-6df6-4b32-93a5-bd6c94ca2281 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:02.657897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dab5e-18ef-4fea-8962-c5a7fe89367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:02.658749 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dab5e-18ef-4fea-8962-c5a7fe89367c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.659247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dab5e-18ef-4fea-8962-c5a7fe89367c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.659742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dab5e-18ef-4fea-8962-c5a7fe89367c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.660935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dab5e-18ef-4fea-8962-c5a7fe89367c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:02.663251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dab5e-18ef-4fea-8962-c5a7fe89367c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.688631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:02.697088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1142/2289] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:26:02 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:02.701473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013df9ca-06f0-4e10-bd30-cb467ecb67ce None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:02.712618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013df9ca-06f0-4e10-bd30-cb467ecb67ce None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:02.732195 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:02.774778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570fad6-1654-4106-a2c1-782ddc79e614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:02.775380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570fad6-1654-4106-a2c1-782ddc79e614 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.775658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570fad6-1654-4106-a2c1-782ddc79e614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.775934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570fad6-1654-4106-a2c1-782ddc79e614 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.776635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570fad6-1654-4106-a2c1-782ddc79e614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:02.778578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570fad6-1654-4106-a2c1-782ddc79e614 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.802179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1148/2290] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:26:02 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:02.806070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dab5e-18ef-4fea-8962-c5a7fe89367c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:02.817325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dab5e-18ef-4fea-8962-c5a7fe89367c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:02.822740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a833e4-fd9e-4a04-82e2-2b74ee74f759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:02.824696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a833e4-fd9e-4a04-82e2-2b74ee74f759 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.824979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a833e4-fd9e-4a04-82e2-2b74ee74f759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.825251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a833e4-fd9e-4a04-82e2-2b74ee74f759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.825905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a833e4-fd9e-4a04-82e2-2b74ee74f759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:02.827550 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a833e4-fd9e-4a04-82e2-2b74ee74f759 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.839551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1143/2291] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:26:02.844537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570fad6-1654-4106-a2c1-782ddc79e614 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:02.854997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570fad6-1654-4106-a2c1-782ddc79e614 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:02.859596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d038d2d-d20d-4934-bd7a-c9a6a7477bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:02.860477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d038d2d-d20d-4934-bd7a-c9a6a7477bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.860760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d038d2d-d20d-4934-bd7a-c9a6a7477bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.861034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d038d2d-d20d-4934-bd7a-c9a6a7477bdc admin admin] PATH_INFO: `/v3/projects/fd24de1bcf7c4ccd920ab0a9dcf139e5/users/e813bf8087a745478ec0e9a8b6256c80/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.877589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d038d2d-d20d-4934-bd7a-c9a6a7477bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd24de1bcf7c4ccd920ab0a9dcf139e5, user_id=e813bf8087a745478ec0e9a8b6256c80, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:02.880414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d038d2d-d20d-4934-bd7a-c9a6a7477bdc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.894087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1149/2292] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:02 2022] PUT /identity/v3/projects/fd24de1bcf7c4ccd920ab0a9dcf139e5/users/e813bf8087a745478ec0e9a8b6256c80/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:02.898336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a833e4-fd9e-4a04-82e2-2b74ee74f759 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:02.911297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a833e4-fd9e-4a04-82e2-2b74ee74f759 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:02.977979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac11a2-2c3f-4cad-b409-219890cf719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:02.978605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac11a2-2c3f-4cad-b409-219890cf719f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:02.978878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac11a2-2c3f-4cad-b409-219890cf719f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:02.979656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac11a2-2c3f-4cad-b409-219890cf719f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:02.980323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac11a2-2c3f-4cad-b409-219890cf719f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:02.982086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac11a2-2c3f-4cad-b409-219890cf719f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:02.992108 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1144/2293] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:26:02.997286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d038d2d-d20d-4934-bd7a-c9a6a7477bdc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:03.009572 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d038d2d-d20d-4934-bd7a-c9a6a7477bdc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:03.017296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd943bc-208f-4c0e-a86d-e3d59f1436b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:03.018388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd943bc-208f-4c0e-a86d-e3d59f1436b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:03.018847 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd943bc-208f-4c0e-a86d-e3d59f1436b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:03.019123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd943bc-208f-4c0e-a86d-e3d59f1436b2 admin admin] PATH_INFO: `/v3/projects/fd24de1bcf7c4ccd920ab0a9dcf139e5/users/e813bf8087a745478ec0e9a8b6256c80/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:03.021757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd943bc-208f-4c0e-a86d-e3d59f1436b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd24de1bcf7c4ccd920ab0a9dcf139e5, user_id=e813bf8087a745478ec0e9a8b6256c80, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:03.027697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd943bc-208f-4c0e-a86d-e3d59f1436b2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:03.047240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1150/2294] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:02 2022] PUT /identity/v3/projects/fd24de1bcf7c4ccd920ab0a9dcf139e5/users/e813bf8087a745478ec0e9a8b6256c80/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:03.060042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d57347-9c3d-4fd1-ab74-cf221ee4ce3d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:03.060525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d57347-9c3d-4fd1-ab74-cf221ee4ce3d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:03.060987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d57347-9c3d-4fd1-ab74-cf221ee4ce3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:03.105880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f0d57347-9c3d-4fd1-ab74-cf221ee4ce3d None None] MFA Rules not processed for user `e813bf8087a745478ec0e9a8b6256c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:03.107974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f0d57347-9c3d-4fd1-ab74-cf221ee4ce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:03.210251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1145/2295] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:03.222840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd943bc-208f-4c0e-a86d-e3d59f1436b2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:03.224751 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5cd943bc-208f-4c0e-a86d-e3d59f1436b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:03.292367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd943bc-208f-4c0e-a86d-e3d59f1436b2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:03.296530 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0eab4-7987-4868-bea6-fdc8b1d77664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:03.297153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0eab4-7987-4868-bea6-fdc8b1d77664 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:03.297433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0eab4-7987-4868-bea6-fdc8b1d77664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:03.297707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0eab4-7987-4868-bea6-fdc8b1d77664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:03.308889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0eab4-7987-4868-bea6-fdc8b1d77664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:03.310354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0eab4-7987-4868-bea6-fdc8b1d77664 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:03.322051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1151/2296] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:26:03.776942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d57347-9c3d-4fd1-ab74-cf221ee4ce3d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:03.845015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d57347-9c3d-4fd1-ab74-cf221ee4ce3d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:03.849516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:03.850132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:03.850413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:03.850683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] PATH_INFO: `/v3/users/f4bb09d60d244901a236662d6e443282` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:03.851800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4bb09d60d244901a236662d6e443282)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:03.853890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:03.867155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:03.905292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4bb09d60d244901a236662d6e443282'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:03.912800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4bb09d60d244901a236662d6e443282'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:03.913320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4bb09d60d244901a236662d6e443282'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:03.927039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4bb09d60d244901a236662d6e443282'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:03.935756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4bb09d60d244901a236662d6e443282'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:03.941902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1146/2297] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:26:03 2022] DELETE /identity/v3/users/f4bb09d60d244901a236662d6e443282 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:03.946848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df19a-e8a2-4d7a-aeeb-0a187f943e04 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:03.947446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df19a-e8a2-4d7a-aeeb-0a187f943e04 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:03.947565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df19a-e8a2-4d7a-aeeb-0a187f943e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:03.969002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-af2df19a-e8a2-4d7a-aeeb-0a187f943e04 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:03.970202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-af2df19a-e8a2-4d7a-aeeb-0a187f943e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:04.055999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1152/2298] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:04.065510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:04.067280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:04.158464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e265d-c886-4c82-ac1d-fbde169e135c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:04.162602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574ed1-ad0b-4fe8-8eb5-6f9df5819f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:04.163216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574ed1-ad0b-4fe8-8eb5-6f9df5819f02 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:04.163500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574ed1-ad0b-4fe8-8eb5-6f9df5819f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:04.163781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29574ed1-ad0b-4fe8-8eb5-6f9df5819f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:04.173642 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574ed1-ad0b-4fe8-8eb5-6f9df5819f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:04.175083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29574ed1-ad0b-4fe8-8eb5-6f9df5819f02 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:04.185344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1147/2299] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:04.503141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2df19a-e8a2-4d7a-aeeb-0a187f943e04 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:04.505187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-af2df19a-e8a2-4d7a-aeeb-0a187f943e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:04.520492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2df19a-e8a2-4d7a-aeeb-0a187f943e04 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:04.527656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:04.528377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:04.528793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:04.529072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] PATH_INFO: `/v3/projects/b52ac5475a7643329ecfc06cca6e8f73` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:04.531524 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52ac5475a7643329ecfc06cca6e8f73)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:04.533887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:04.555125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:04.625427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52ac5475a7643329ecfc06cca6e8f73', 'request_id': 'req-680b8782-dd7a-4e6d-b289-135ca444bbf9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.639412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52ac5475a7643329ecfc06cca6e8f73', 'request_id': 'req-680b8782-dd7a-4e6d-b289-135ca444bbf9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.650429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52ac5475a7643329ecfc06cca6e8f73', 'request_id': 'req-680b8782-dd7a-4e6d-b289-135ca444bbf9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.670502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] The token cache is being invalidate because project b52ac5475a7643329ecfc06cca6e8f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:26:04.671629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.672455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-680b8782-dd7a-4e6d-b289-135ca444bbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.674013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1153/2300] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:26:04 2022] DELETE /identity/v3/projects/b52ac5475a7643329ecfc06cca6e8f73 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:04.679898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574ed1-ad0b-4fe8-8eb5-6f9df5819f02 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:04.695296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29574ed1-ad0b-4fe8-8eb5-6f9df5819f02 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:04.785872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:04.786582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:04.786864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:04.787096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] PATH_INFO: `/v3/users/e813bf8087a745478ec0e9a8b6256c80` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:04.788369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=e813bf8087a745478ec0e9a8b6256c80)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:04.791005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:04.856586 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:04.872687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e813bf8087a745478ec0e9a8b6256c80'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.880657 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e813bf8087a745478ec0e9a8b6256c80'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.889200 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e813bf8087a745478ec0e9a8b6256c80'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.904783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e813bf8087a745478ec0e9a8b6256c80'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.912769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e813bf8087a745478ec0e9a8b6256c80'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:04.919349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1148/2301] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:26:04 2022] DELETE /identity/v3/users/e813bf8087a745478ec0e9a8b6256c80 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:05.173025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbb9d1-2476-4e6a-ab4c-7c9178b05cfb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:05.173946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbb9d1-2476-4e6a-ab4c-7c9178b05cfb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:05.174289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbb9d1-2476-4e6a-ab4c-7c9178b05cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:05.219326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f6dbb9d1-2476-4e6a-ab4c-7c9178b05cfb None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:05.221580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f6dbb9d1-2476-4e6a-ab4c-7c9178b05cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:05.339910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:05.341761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:05.392963 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1154/2302] 10.222.0.75 () {58 vars in 967 bytes} [Fri Aug 19 15:26:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:05.410961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbb9d1-2476-4e6a-ab4c-7c9178b05cfb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:05.420399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f6dbb9d1-2476-4e6a-ab4c-7c9178b05cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:05.488324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a4dda-4ef7-4d0a-b0e3-0bcec7b88ded admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:05.493895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:05.494534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:05.495338 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:05.495654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] PATH_INFO: `/v3/projects/fd24de1bcf7c4ccd920ab0a9dcf139e5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:05.497563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd24de1bcf7c4ccd920ab0a9dcf139e5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:05.501415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:05.581653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbb9d1-2476-4e6a-ab4c-7c9178b05cfb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:05.589788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd24de1bcf7c4ccd920ab0a9dcf139e5', 'request_id': 'req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:05.599307 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd24de1bcf7c4ccd920ab0a9dcf139e5', 'request_id': 'req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:05.608841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd24de1bcf7c4ccd920ab0a9dcf139e5', 'request_id': 'req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:05.627428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] The token cache is being invalidate because project fd24de1bcf7c4ccd920ab0a9dcf139e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:26:05.627690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:05.628356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0b66ceae-3d78-4c7b-976e-ed2550e06dd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:05.630180 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1149/2303] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:26:05 2022] DELETE /identity/v3/projects/fd24de1bcf7c4ccd920ab0a9dcf139e5 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:26:05.635094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb87c60-df53-487b-8b13-e7a2f9078fda None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:05.635426 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb87c60-df53-487b-8b13-e7a2f9078fda None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:05.636350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb87c60-df53-487b-8b13-e7a2f9078fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:05.644424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:05.660978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-dbb87c60-df53-487b-8b13-e7a2f9078fda None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:05.662313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dbb87c60-df53-487b-8b13-e7a2f9078fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:05.665657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e433b53-feec-4be3-9a5a-f0527f7b7e97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:05.666277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e433b53-feec-4be3-9a5a-f0527f7b7e97 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:05.666562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e433b53-feec-4be3-9a5a-f0527f7b7e97 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:05.667814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e433b53-feec-4be3-9a5a-f0527f7b7e97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:05.670934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e433b53-feec-4be3-9a5a-f0527f7b7e97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:05.701366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:05.732574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1150/2304] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:26:05.741937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21b4c9-70f5-4e6b-b7f3-5aab336d3d85 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:05.742254 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21b4c9-70f5-4e6b-b7f3-5aab336d3d85 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:05.742488 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21b4c9-70f5-4e6b-b7f3-5aab336d3d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:05.750164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e433b53-feec-4be3-9a5a-f0527f7b7e97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:05.752067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e433b53-feec-4be3-9a5a-f0527f7b7e97 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:05.761500 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1155/2305] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:26:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 15:26:05.764111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3e21b4c9-70f5-4e6b-b7f3-5aab336d3d85 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:05.765558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3e21b4c9-70f5-4e6b-b7f3-5aab336d3d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:05.775133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1151/2306] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:26:05.780000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e433b53-feec-4be3-9a5a-f0527f7b7e97 service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:05.795304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e433b53-feec-4be3-9a5a-f0527f7b7e97 service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:05.801347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febafbc-81d4-4257-8bbd-23f1ad82aa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:05.801645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febafbc-81d4-4257-8bbd-23f1ad82aa47 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:05.801939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febafbc-81d4-4257-8bbd-23f1ad82aa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:05.802214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febafbc-81d4-4257-8bbd-23f1ad82aa47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:05.802880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febafbc-81d4-4257-8bbd-23f1ad82aa47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:26:05.803190 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febafbc-81d4-4257-8bbd-23f1ad82aa47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:05.805150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febafbc-81d4-4257-8bbd-23f1ad82aa47 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:05.818848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1156/2307] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:26:05.823989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21b4c9-70f5-4e6b-b7f3-5aab336d3d85 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:05.833947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21b4c9-70f5-4e6b-b7f3-5aab336d3d85 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:05.843028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5783c-a4ef-4732-a5a5-2f664d5540ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:05.843809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5783c-a4ef-4732-a5a5-2f664d5540ad admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:05.844156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5783c-a4ef-4732-a5a5-2f664d5540ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:05.844802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5783c-a4ef-4732-a5a5-2f664d5540ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:05.845425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5783c-a4ef-4732-a5a5-2f664d5540ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:05.847710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5783c-a4ef-4732-a5a5-2f664d5540ad admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:05.888886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1152/2308] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:26:05 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:26:05.892337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febafbc-81d4-4257-8bbd-23f1ad82aa47 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:05.921340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febafbc-81d4-4257-8bbd-23f1ad82aa47 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.020011 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d898377-331b-4d94-87c8-4ac7b34fc0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.020692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d898377-331b-4d94-87c8-4ac7b34fc0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.020975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d898377-331b-4d94-87c8-4ac7b34fc0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.021403 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d898377-331b-4d94-87c8-4ac7b34fc0d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.021892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d898377-331b-4d94-87c8-4ac7b34fc0d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.024671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d898377-331b-4d94-87c8-4ac7b34fc0d3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.057084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1157/2309] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:26:05 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:06.062026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5783c-a4ef-4732-a5a5-2f664d5540ad admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.075185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5783c-a4ef-4732-a5a5-2f664d5540ad admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.083431 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a99b6b-0f94-4008-b97f-bac3c78c1910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.084246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a99b6b-0f94-4008-b97f-bac3c78c1910 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.084678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a99b6b-0f94-4008-b97f-bac3c78c1910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.084929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a99b6b-0f94-4008-b97f-bac3c78c1910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.085538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a99b6b-0f94-4008-b97f-bac3c78c1910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.087158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a99b6b-0f94-4008-b97f-bac3c78c1910 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.114170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1153/2310] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:06.125789 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d898377-331b-4d94-87c8-4ac7b34fc0d3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.149467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d898377-331b-4d94-87c8-4ac7b34fc0d3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.155889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df239a0-b093-4ff9-ae0f-9abde66f1d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.156821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df239a0-b093-4ff9-ae0f-9abde66f1d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.157211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df239a0-b093-4ff9-ae0f-9abde66f1d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.157611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df239a0-b093-4ff9-ae0f-9abde66f1d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.158355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df239a0-b093-4ff9-ae0f-9abde66f1d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.162649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df239a0-b093-4ff9-ae0f-9abde66f1d6f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.176532 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1158/2311] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:06.182483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a99b6b-0f94-4008-b97f-bac3c78c1910 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.194985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a99b6b-0f94-4008-b97f-bac3c78c1910 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.200672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555a077-683b-43d6-b04a-69f55cdac888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.201471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555a077-683b-43d6-b04a-69f55cdac888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.201879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555a077-683b-43d6-b04a-69f55cdac888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.202256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555a077-683b-43d6-b04a-69f55cdac888 admin admin] PATH_INFO: `/v3/projects/3c172098ff4748af8616d345563a6b0f/users/fab445217bbb43a2b047fdc9409695dd/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.227327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555a077-683b-43d6-b04a-69f55cdac888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c172098ff4748af8616d345563a6b0f, user_id=fab445217bbb43a2b047fdc9409695dd, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.231430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555a077-683b-43d6-b04a-69f55cdac888 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.263914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1154/2312] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:06 2022] PUT /identity/v3/projects/3c172098ff4748af8616d345563a6b0f/users/fab445217bbb43a2b047fdc9409695dd/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:06.275636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0551ea-081c-436e-bc67-c23934335e1c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.276681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0551ea-081c-436e-bc67-c23934335e1c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.277118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0551ea-081c-436e-bc67-c23934335e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.308183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-db0551ea-081c-436e-bc67-c23934335e1c None None] MFA Rules not processed for user `fab445217bbb43a2b047fdc9409695dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:06.309538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-db0551ea-081c-436e-bc67-c23934335e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:06.366961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:06.391287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1159/2313] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:06.395951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555a077-683b-43d6-b04a-69f55cdac888 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.465567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555a077-683b-43d6-b04a-69f55cdac888 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.471244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d761ed-ee21-4664-aa91-3fa7c6f1bbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.471878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d761ed-ee21-4664-aa91-3fa7c6f1bbef admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.472162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d761ed-ee21-4664-aa91-3fa7c6f1bbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.472468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d761ed-ee21-4664-aa91-3fa7c6f1bbef admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.473294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d761ed-ee21-4664-aa91-3fa7c6f1bbef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.475268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d761ed-ee21-4664-aa91-3fa7c6f1bbef admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.507031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1155/2314] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:26:06 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:06.508163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0551ea-081c-436e-bc67-c23934335e1c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.520025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0551ea-081c-436e-bc67-c23934335e1c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.581007 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d925ee0-f4c0-4041-9553-62327763029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.581603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d925ee0-f4c0-4041-9553-62327763029e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.581877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d925ee0-f4c0-4041-9553-62327763029e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.582144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d925ee0-f4c0-4041-9553-62327763029e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.582784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d925ee0-f4c0-4041-9553-62327763029e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.584759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d925ee0-f4c0-4041-9553-62327763029e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.613414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1160/2315] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:26:06 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:06.617717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d761ed-ee21-4664-aa91-3fa7c6f1bbef admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.629037 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d761ed-ee21-4664-aa91-3fa7c6f1bbef admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.633821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25162197-4eff-4ff1-ab41-de85630f3cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.634430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25162197-4eff-4ff1-ab41-de85630f3cff admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.634712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25162197-4eff-4ff1-ab41-de85630f3cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.634988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25162197-4eff-4ff1-ab41-de85630f3cff admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.636256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25162197-4eff-4ff1-ab41-de85630f3cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.637927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25162197-4eff-4ff1-ab41-de85630f3cff admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.650547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1156/2316] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:06.655533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d925ee0-f4c0-4041-9553-62327763029e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.665697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d925ee0-f4c0-4041-9553-62327763029e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.671326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5048632-cbf3-41a8-a4e3-a573e46e9a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.672000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5048632-cbf3-41a8-a4e3-a573e46e9a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.672275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5048632-cbf3-41a8-a4e3-a573e46e9a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.672635 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5048632-cbf3-41a8-a4e3-a573e46e9a73 admin admin] PATH_INFO: `/v3/projects/a18d09eb9f4a4d30906e4cbc02c6c646/users/64681958f8b944ce91aefe633bac90ab/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.698778 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5048632-cbf3-41a8-a4e3-a573e46e9a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18d09eb9f4a4d30906e4cbc02c6c646, user_id=64681958f8b944ce91aefe633bac90ab, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.701580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5048632-cbf3-41a8-a4e3-a573e46e9a73 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.715669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1161/2317] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:06 2022] PUT /identity/v3/projects/a18d09eb9f4a4d30906e4cbc02c6c646/users/64681958f8b944ce91aefe633bac90ab/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:06.719703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25162197-4eff-4ff1-ab41-de85630f3cff admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.731228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25162197-4eff-4ff1-ab41-de85630f3cff admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.779706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:06.792448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da0e66-a8db-44a5-910b-739b90109074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.793069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da0e66-a8db-44a5-910b-739b90109074 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.793352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da0e66-a8db-44a5-910b-739b90109074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.793625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da0e66-a8db-44a5-910b-739b90109074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.794544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da0e66-a8db-44a5-910b-739b90109074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.796206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da0e66-a8db-44a5-910b-739b90109074 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.808887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1157/2318] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:06.813084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5048632-cbf3-41a8-a4e3-a573e46e9a73 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.823813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5048632-cbf3-41a8-a4e3-a573e46e9a73 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:06.828755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e7407-bb50-4930-891d-736a1ae7b07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:06.829421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e7407-bb50-4930-891d-736a1ae7b07e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.829701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e7407-bb50-4930-891d-736a1ae7b07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.829976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e7407-bb50-4930-891d-736a1ae7b07e admin admin] PATH_INFO: `/v3/projects/a18d09eb9f4a4d30906e4cbc02c6c646/users/64681958f8b944ce91aefe633bac90ab/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.831883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e7407-bb50-4930-891d-736a1ae7b07e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18d09eb9f4a4d30906e4cbc02c6c646, user_id=64681958f8b944ce91aefe633bac90ab, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:06.834589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e7407-bb50-4930-891d-736a1ae7b07e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:06.846645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1162/2319] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:06 2022] PUT /identity/v3/projects/a18d09eb9f4a4d30906e4cbc02c6c646/users/64681958f8b944ce91aefe633bac90ab/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:06.858374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f9bd-669f-44ab-a355-72651ea19e05 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:06.858906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f9bd-669f-44ab-a355-72651ea19e05 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:06.858906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f9bd-669f-44ab-a355-72651ea19e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:06.889626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e639f9bd-669f-44ab-a355-72651ea19e05 None None] MFA Rules not processed for user `64681958f8b944ce91aefe633bac90ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:06.890786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e639f9bd-669f-44ab-a355-72651ea19e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:06.983503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1158/2320] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:06.998173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e7407-bb50-4930-891d-736a1ae7b07e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:06.999907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-392e7407-bb50-4930-891d-736a1ae7b07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:07.065623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e7407-bb50-4930-891d-736a1ae7b07e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:07.069634 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984661b-6c2e-435d-8c6a-91627f39e267 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:07.070259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984661b-6c2e-435d-8c6a-91627f39e267 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:07.070685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984661b-6c2e-435d-8c6a-91627f39e267 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:07.070968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984661b-6c2e-435d-8c6a-91627f39e267 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:07.081007 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984661b-6c2e-435d-8c6a-91627f39e267 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:07.082577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984661b-6c2e-435d-8c6a-91627f39e267 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:07.098860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1163/2321] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:07.247806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639f9bd-669f-44ab-a355-72651ea19e05 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:07.262537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639f9bd-669f-44ab-a355-72651ea19e05 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:07.267486 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bb6df-5afd-4bd1-8edd-dfbe3045fad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:07.268241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bb6df-5afd-4bd1-8edd-dfbe3045fad3 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:07.268656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bb6df-5afd-4bd1-8edd-dfbe3045fad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:07.269886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bb6df-5afd-4bd1-8edd-dfbe3045fad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:07.357199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bb6df-5afd-4bd1-8edd-dfbe3045fad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:07.358849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318bb6df-5afd-4bd1-8edd-dfbe3045fad3 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:07.375770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1159/2322] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:07.691739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984661b-6c2e-435d-8c6a-91627f39e267 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:07.736529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:07.759280 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984661b-6c2e-435d-8c6a-91627f39e267 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:07.764585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:07.765229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:07.765510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:07.765788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] PATH_INFO: `/v3/users/fab445217bbb43a2b047fdc9409695dd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:07.766921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fab445217bbb43a2b047fdc9409695dd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:07.768916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:07.827035 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fab445217bbb43a2b047fdc9409695dd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:07.828002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fab445217bbb43a2b047fdc9409695dd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:07.828944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fab445217bbb43a2b047fdc9409695dd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:07.837379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fab445217bbb43a2b047fdc9409695dd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:07.859168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fab445217bbb43a2b047fdc9409695dd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:07.873236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1164/2323] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:26:07 2022] DELETE /identity/v3/users/fab445217bbb43a2b047fdc9409695dd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:07.878480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64c276-d853-4666-8b00-f82dc0716264 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:07.879019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64c276-d853-4666-8b00-f82dc0716264 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:07.879019 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64c276-d853-4666-8b00-f82dc0716264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:07.903363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7b64c276-d853-4666-8b00-f82dc0716264 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:07.904535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7b64c276-d853-4666-8b00-f82dc0716264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:07.970641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1160/2324] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:07.982546 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:07.984723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:08.061161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c7a13-1edb-4015-aac6-0bd453ebf07d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:08.065036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe7d81-a4d7-4b81-b207-def8fb59bb0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:08.065656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe7d81-a4d7-4b81-b207-def8fb59bb0c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:08.065938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe7d81-a4d7-4b81-b207-def8fb59bb0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:08.066215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe7d81-a4d7-4b81-b207-def8fb59bb0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:08.082481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe7d81-a4d7-4b81-b207-def8fb59bb0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:08.084460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe7d81-a4d7-4b81-b207-def8fb59bb0c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:08.096072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1165/2325] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:08.408421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64c276-d853-4666-8b00-f82dc0716264 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:08.410228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7b64c276-d853-4666-8b00-f82dc0716264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:08.424841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64c276-d853-4666-8b00-f82dc0716264 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:08.439890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:08.440587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:08.440958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:08.441356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] PATH_INFO: `/v3/projects/3c172098ff4748af8616d345563a6b0f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:08.442769 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c172098ff4748af8616d345563a6b0f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:08.445245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:08.453360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:08.514003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c172098ff4748af8616d345563a6b0f', 'request_id': 'req-3b685084-b922-4ae1-a5cb-91ab8c89918a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.522033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c172098ff4748af8616d345563a6b0f', 'request_id': 'req-3b685084-b922-4ae1-a5cb-91ab8c89918a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.530929 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c172098ff4748af8616d345563a6b0f', 'request_id': 'req-3b685084-b922-4ae1-a5cb-91ab8c89918a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.549047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] The token cache is being invalidate because project 3c172098ff4748af8616d345563a6b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:26:08.549549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.550547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.553145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1161/2326] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:26:08 2022] DELETE /identity/v3/projects/3c172098ff4748af8616d345563a6b0f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:26:08.559724 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe7d81-a4d7-4b81-b207-def8fb59bb0c service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:08.570089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe7d81-a4d7-4b81-b207-def8fb59bb0c service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:08.639244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:08.639889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:08.640170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:08.640573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] PATH_INFO: `/v3/users/64681958f8b944ce91aefe633bac90ab` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:08.641714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64681958f8b944ce91aefe633bac90ab)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:08.643729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:08.646706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:08.715761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64681958f8b944ce91aefe633bac90ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.716605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64681958f8b944ce91aefe633bac90ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.717291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64681958f8b944ce91aefe633bac90ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.725869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64681958f8b944ce91aefe633bac90ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.739299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d02ae863-2493-4e51-bdf5-4396031ae4f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64681958f8b944ce91aefe633bac90ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:08.762490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1166/2327] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:26:08 2022] DELETE /identity/v3/users/64681958f8b944ce91aefe633bac90ab => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:09.127480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:09.129961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:09.147396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b685084-b922-4ae1-a5cb-91ab8c89918a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:09.248071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:09.249630 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:09.250100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:09.251615 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] PATH_INFO: `/v3/projects/a18d09eb9f4a4d30906e4cbc02c6c646` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:09.254344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18d09eb9f4a4d30906e4cbc02c6c646)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:09.260588 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:09.362188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18d09eb9f4a4d30906e4cbc02c6c646', 'request_id': 'req-f560360e-9714-4c70-8b04-6fe76613fc6f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:09.369727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18d09eb9f4a4d30906e4cbc02c6c646', 'request_id': 'req-f560360e-9714-4c70-8b04-6fe76613fc6f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:09.383038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18d09eb9f4a4d30906e4cbc02c6c646', 'request_id': 'req-f560360e-9714-4c70-8b04-6fe76613fc6f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:09.406128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] The token cache is being invalidate because project a18d09eb9f4a4d30906e4cbc02c6c646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:26:09.406128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:09.406128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f560360e-9714-4c70-8b04-6fe76613fc6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:09.406128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1162/2328] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:26:09 2022] DELETE /identity/v3/projects/a18d09eb9f4a4d30906e4cbc02c6c646 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:09.410354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96357124-fca7-4b2e-a109-ef24de3e1c64 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:09.410645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96357124-fca7-4b2e-a109-ef24de3e1c64 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:09.410925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96357124-fca7-4b2e-a109-ef24de3e1c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:09.441395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-96357124-fca7-4b2e-a109-ef24de3e1c64 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:09.443559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-96357124-fca7-4b2e-a109-ef24de3e1c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:09.561436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1167/2329] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:09.637429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c1c80-f30d-4acb-9760-05d81451a354 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:09.637698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c1c80-f30d-4acb-9760-05d81451a354 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:09.637972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c1c80-f30d-4acb-9760-05d81451a354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:09.660678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-745c1c80-f30d-4acb-9760-05d81451a354 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:09.661918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-745c1c80-f30d-4acb-9760-05d81451a354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:09.676782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1163/2330] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:09.683228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96357124-fca7-4b2e-a109-ef24de3e1c64 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:09.693708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96357124-fca7-4b2e-a109-ef24de3e1c64 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:09.701796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc7db9-540b-4d40-8463-c7745f08be62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:09.701796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc7db9-540b-4d40-8463-c7745f08be62 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:09.701796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc7db9-540b-4d40-8463-c7745f08be62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:09.701796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc7db9-540b-4d40-8463-c7745f08be62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:09.702264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc7db9-540b-4d40-8463-c7745f08be62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:26:09.702264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc7db9-540b-4d40-8463-c7745f08be62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:09.702814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc7db9-540b-4d40-8463-c7745f08be62 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:09.719182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1168/2331] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:26:09.724286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c1c80-f30d-4acb-9760-05d81451a354 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:09.741296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c1c80-f30d-4acb-9760-05d81451a354 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:09.741296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69248238-aa63-4c5a-9fc9-a63c7a90e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:09.741296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69248238-aa63-4c5a-9fc9-a63c7a90e8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:09.741574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69248238-aa63-4c5a-9fc9-a63c7a90e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:09.741781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69248238-aa63-4c5a-9fc9-a63c7a90e8d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:09.742560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69248238-aa63-4c5a-9fc9-a63c7a90e8d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:09.744545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69248238-aa63-4c5a-9fc9-a63c7a90e8d2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:09.771813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:09.806876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1164/2332] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:26:09 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:09.818066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc7db9-540b-4d40-8463-c7745f08be62 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:09.849985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc7db9-540b-4d40-8463-c7745f08be62 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:09.971066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64175a9b-4439-4732-9da6-2fdacd63accf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:09.971974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64175a9b-4439-4732-9da6-2fdacd63accf admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:09.972895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64175a9b-4439-4732-9da6-2fdacd63accf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:09.974052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64175a9b-4439-4732-9da6-2fdacd63accf admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:09.974948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64175a9b-4439-4732-9da6-2fdacd63accf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:09.978998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64175a9b-4439-4732-9da6-2fdacd63accf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.019695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:10.031928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1169/2333] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:26:09 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:10.035972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69248238-aa63-4c5a-9fc9-a63c7a90e8d2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.052835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69248238-aa63-4c5a-9fc9-a63c7a90e8d2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:10.064348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4fd74-aaa1-42c0-aecf-c891af14f09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:10.064679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4fd74-aaa1-42c0-aecf-c891af14f09e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.065888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4fd74-aaa1-42c0-aecf-c891af14f09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.066214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4fd74-aaa1-42c0-aecf-c891af14f09e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.067276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4fd74-aaa1-42c0-aecf-c891af14f09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:10.071196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4fd74-aaa1-42c0-aecf-c891af14f09e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.109507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1165/2334] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:26:10.114448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64175a9b-4439-4732-9da6-2fdacd63accf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.129150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64175a9b-4439-4732-9da6-2fdacd63accf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:10.139627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca36ef0-2e0b-4fa8-9464-6a5aad86d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:10.140449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca36ef0-2e0b-4fa8-9464-6a5aad86d954 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.140825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca36ef0-2e0b-4fa8-9464-6a5aad86d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.141226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca36ef0-2e0b-4fa8-9464-6a5aad86d954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.142088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca36ef0-2e0b-4fa8-9464-6a5aad86d954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:10.144422 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca36ef0-2e0b-4fa8-9464-6a5aad86d954 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.160735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1170/2335] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:10.167742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4fd74-aaa1-42c0-aecf-c891af14f09e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.179281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4fd74-aaa1-42c0-aecf-c891af14f09e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:10.185092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc56af-e76c-4192-b966-81737c9f3830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:10.185903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc56af-e76c-4192-b966-81737c9f3830 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.186248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc56af-e76c-4192-b966-81737c9f3830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.186587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc56af-e76c-4192-b966-81737c9f3830 admin admin] PATH_INFO: `/v3/projects/ce54331eb3d34946865d569b51800674/users/cf681cdbc80e441b888b0214be3b4d90/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.205572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc56af-e76c-4192-b966-81737c9f3830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce54331eb3d34946865d569b51800674, user_id=cf681cdbc80e441b888b0214be3b4d90, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:10.208482 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc56af-e76c-4192-b966-81737c9f3830 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.227026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1166/2336] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:10 2022] PUT /identity/v3/projects/ce54331eb3d34946865d569b51800674/users/cf681cdbc80e441b888b0214be3b4d90/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:10.239685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fed549-8194-4b7d-952c-6cd2d5dd1e4e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.240699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fed549-8194-4b7d-952c-6cd2d5dd1e4e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.241339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fed549-8194-4b7d-952c-6cd2d5dd1e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.281953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-72fed549-8194-4b7d-952c-6cd2d5dd1e4e None None] MFA Rules not processed for user `cf681cdbc80e441b888b0214be3b4d90`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:10.283237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-72fed549-8194-4b7d-952c-6cd2d5dd1e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:10.362783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1171/2337] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:10.367440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc56af-e76c-4192-b966-81737c9f3830 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.446403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc56af-e76c-4192-b966-81737c9f3830 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:10.455397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228ed03-8635-49c8-87da-061ef0df1747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:10.456208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228ed03-8635-49c8-87da-061ef0df1747 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.456606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228ed03-8635-49c8-87da-061ef0df1747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.456885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228ed03-8635-49c8-87da-061ef0df1747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.457545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228ed03-8635-49c8-87da-061ef0df1747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:10.459532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228ed03-8635-49c8-87da-061ef0df1747 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.497782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1167/2338] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:26:10 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:26:10.501719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fed549-8194-4b7d-952c-6cd2d5dd1e4e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.514248 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fed549-8194-4b7d-952c-6cd2d5dd1e4e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:10.612036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5375c3-dd35-4928-92ff-1b08e398a001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:10.613169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5375c3-dd35-4928-92ff-1b08e398a001 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.613651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5375c3-dd35-4928-92ff-1b08e398a001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.614124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5375c3-dd35-4928-92ff-1b08e398a001 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.614952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5375c3-dd35-4928-92ff-1b08e398a001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:10.617310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5375c3-dd35-4928-92ff-1b08e398a001 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.646678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1172/2339] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:26:10 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:10.651603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228ed03-8635-49c8-87da-061ef0df1747 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.664308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228ed03-8635-49c8-87da-061ef0df1747 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:10.670137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b969a7-98d6-4332-bee1-85edd8bc1d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:10.670756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b969a7-98d6-4332-bee1-85edd8bc1d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.671662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b969a7-98d6-4332-bee1-85edd8bc1d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.672097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b969a7-98d6-4332-bee1-85edd8bc1d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.672833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b969a7-98d6-4332-bee1-85edd8bc1d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:10.674856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b969a7-98d6-4332-bee1-85edd8bc1d73 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.694457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1168/2340] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:10.699454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5375c3-dd35-4928-92ff-1b08e398a001 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.719982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5375c3-dd35-4928-92ff-1b08e398a001 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:10.732232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b969a7-98d6-4332-bee1-85edd8bc1d73 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.733408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d959b9-0bd3-4c1b-a5d9-56f7c88330b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:10.734044 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d959b9-0bd3-4c1b-a5d9-56f7c88330b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.734332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d959b9-0bd3-4c1b-a5d9-56f7c88330b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.734597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d959b9-0bd3-4c1b-a5d9-56f7c88330b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.735254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d959b9-0bd3-4c1b-a5d9-56f7c88330b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:10.736561 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d1b969a7-98d6-4332-bee1-85edd8bc1d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:10.736934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d959b9-0bd3-4c1b-a5d9-56f7c88330b8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.752627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b969a7-98d6-4332-bee1-85edd8bc1d73 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:10.756223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1173/2341] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:10.757531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d959b9-0bd3-4c1b-a5d9-56f7c88330b8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.759364 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:10.759993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.760268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.761215 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] PATH_INFO: `/v3/users/887eceef50b84dde975e15ac9e7da02b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.762742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=887eceef50b84dde975e15ac9e7da02b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:10.764806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.769871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d959b9-0bd3-4c1b-a5d9-56f7c88330b8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:10.778350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9f80a-1772-4f17-aa4f-1561bd58f8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:10.778350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9f80a-1772-4f17-aa4f-1561bd58f8cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.778350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9f80a-1772-4f17-aa4f-1561bd58f8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.778350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9f80a-1772-4f17-aa4f-1561bd58f8cf admin admin] PATH_INFO: `/v3/projects/f45dcc8ac2664c8ea6d0250d9ef3b9da/users/30784403db1743b08851f44a0d75b335/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.792481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9f80a-1772-4f17-aa4f-1561bd58f8cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f45dcc8ac2664c8ea6d0250d9ef3b9da, user_id=30784403db1743b08851f44a0d75b335, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:10.795885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9f80a-1772-4f17-aa4f-1561bd58f8cf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:10.811773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1174/2342] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:10 2022] PUT /identity/v3/projects/f45dcc8ac2664c8ea6d0250d9ef3b9da/users/30784403db1743b08851f44a0d75b335/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:26:10.822262 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ee66f-f8fb-4e6d-8bec-6c161a362b2b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:10.822950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ee66f-f8fb-4e6d-8bec-6c161a362b2b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:10.833661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '887eceef50b84dde975e15ac9e7da02b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:10.833887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ee66f-f8fb-4e6d-8bec-6c161a362b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:10.840192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '887eceef50b84dde975e15ac9e7da02b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:10.841005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '887eceef50b84dde975e15ac9e7da02b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:10.854034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '887eceef50b84dde975e15ac9e7da02b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:10.856352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8d2ee66f-f8fb-4e6d-8bec-6c161a362b2b None None] MFA Rules not processed for user `30784403db1743b08851f44a0d75b335`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:10.857621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8d2ee66f-f8fb-4e6d-8bec-6c161a362b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:10.866780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '887eceef50b84dde975e15ac9e7da02b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:10.877621 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1169/2343] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:26:10 2022] DELETE /identity/v3/users/887eceef50b84dde975e15ac9e7da02b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:10.878286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:10.970594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1175/2344] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:26:10.974715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:10.976963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:10.991281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:11.130617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f2f90-e8ac-4d4c-9b10-9e1bc1d01908 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:11.143417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ee66f-f8fb-4e6d-8bec-6c161a362b2b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:11.149213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8d2ee66f-f8fb-4e6d-8bec-6c161a362b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:11.150756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a886e-6260-4d0f-8443-b4c2a65ac440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:11.151890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a886e-6260-4d0f-8443-b4c2a65ac440 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:11.152093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a886e-6260-4d0f-8443-b4c2a65ac440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:11.156338 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a886e-6260-4d0f-8443-b4c2a65ac440 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:11.157195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a886e-6260-4d0f-8443-b4c2a65ac440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:11.159558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a886e-6260-4d0f-8443-b4c2a65ac440 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:11.216608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1170/2345] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:26:10 2022] POST /identity/v3/projects => generated 375 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:11.221052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a886e-6260-4d0f-8443-b4c2a65ac440 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:11.224440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d07a886e-6260-4d0f-8443-b4c2a65ac440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:11.303131 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ee66f-f8fb-4e6d-8bec-6c161a362b2b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:11.376022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a886e-6260-4d0f-8443-b4c2a65ac440 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:11.383460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6ac6d-1aa7-4112-a7d4-f6f6a9cdfe40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:11.384454 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6ac6d-1aa7-4112-a7d4-f6f6a9cdfe40 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:11.387390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6ac6d-1aa7-4112-a7d4-f6f6a9cdfe40 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:11.387818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6ac6d-1aa7-4112-a7d4-f6f6a9cdfe40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:11.390619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-15a6ac6d-1aa7-4112-a7d4-f6f6a9cdfe40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:11.404620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6ac6d-1aa7-4112-a7d4-f6f6a9cdfe40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:11.404891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:11.405562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:11.405842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:11.406120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] PATH_INFO: `/v3/projects/fc839eae8c504e298673b61d4de8b0e5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:11.406231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6ac6d-1aa7-4112-a7d4-f6f6a9cdfe40 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:11.407306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc839eae8c504e298673b61d4de8b0e5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:11.409850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:11.418527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1171/2346] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:11.419462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6ac6d-1aa7-4112-a7d4-f6f6a9cdfe40 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:11.435129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6ac6d-1aa7-4112-a7d4-f6f6a9cdfe40 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:11.441161 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753947e9-738f-4f86-a0ce-569633b6fba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:11.441785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753947e9-738f-4f86-a0ce-569633b6fba3 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:11.442066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753947e9-738f-4f86-a0ce-569633b6fba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:11.442339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753947e9-738f-4f86-a0ce-569633b6fba3 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:11.443310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753947e9-738f-4f86-a0ce-569633b6fba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:11.445868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753947e9-738f-4f86-a0ce-569633b6fba3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:11.487894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1172/2347] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:26:11 2022] POST /identity/v3/users => generated 294 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:26:11.498778 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753947e9-738f-4f86-a0ce-569633b6fba3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:11.506742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc839eae8c504e298673b61d4de8b0e5', 'request_id': 'req-63fa3423-5bf7-492b-8877-1b669b3274a2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:11.521955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753947e9-738f-4f86-a0ce-569633b6fba3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:11.524124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc839eae8c504e298673b61d4de8b0e5', 'request_id': 'req-63fa3423-5bf7-492b-8877-1b669b3274a2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:11.544539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc839eae8c504e298673b61d4de8b0e5', 'request_id': 'req-63fa3423-5bf7-492b-8877-1b669b3274a2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:11.554523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:11.566375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] The token cache is being invalidate because project fc839eae8c504e298673b61d4de8b0e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:26:11.569107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:11.570638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:11.578598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1176/2348] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:26:11 2022] DELETE /identity/v3/projects/fc839eae8c504e298673b61d4de8b0e5 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:11.702538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8b815-fb71-4966-afd7-dfaf06b64f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:11.704549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8b815-fb71-4966-afd7-dfaf06b64f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:11.705168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8b815-fb71-4966-afd7-dfaf06b64f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:11.705660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8b815-fb71-4966-afd7-dfaf06b64f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:11.707606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8b815-fb71-4966-afd7-dfaf06b64f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:11.709626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8b815-fb71-4966-afd7-dfaf06b64f1c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:11.726674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:11.743757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1173/2349] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:11.752361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:11.754308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:11.787473 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa3423-5bf7-492b-8877-1b669b3274a2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:11.812202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b03388-a55d-4e4a-8717-847a3c3386bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:11.812940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b03388-a55d-4e4a-8717-847a3c3386bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:11.813215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b03388-a55d-4e4a-8717-847a3c3386bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:11.815717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b03388-a55d-4e4a-8717-847a3c3386bf admin admin] PATH_INFO: `/v3/projects/0ee309dd698d4c34a1188132a35c60fc/users/0ea1adaa89ce47b0925e3b0a5611849e/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:11.845884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b03388-a55d-4e4a-8717-847a3c3386bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee309dd698d4c34a1188132a35c60fc, user_id=0ea1adaa89ce47b0925e3b0a5611849e, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:11.848920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b03388-a55d-4e4a-8717-847a3c3386bf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:11.871535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1177/2350] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:11 2022] PUT /identity/v3/projects/0ee309dd698d4c34a1188132a35c60fc/users/0ea1adaa89ce47b0925e3b0a5611849e/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:11.873171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8b815-fb71-4966-afd7-dfaf06b64f1c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:11.892183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8b815-fb71-4966-afd7-dfaf06b64f1c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:12.014324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46274397-5d11-441a-a42b-aa73913a364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:12.014950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46274397-5d11-441a-a42b-aa73913a364c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:12.015225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46274397-5d11-441a-a42b-aa73913a364c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:12.015492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46274397-5d11-441a-a42b-aa73913a364c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:12.016145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46274397-5d11-441a-a42b-aa73913a364c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:12.018033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46274397-5d11-441a-a42b-aa73913a364c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:12.029447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1174/2351] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:26:12.034084 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b03388-a55d-4e4a-8717-847a3c3386bf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:12.055870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b03388-a55d-4e4a-8717-847a3c3386bf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:12.064467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15585a2c-b8ec-4b70-8ae0-0731e670308b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:12.067305 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15585a2c-b8ec-4b70-8ae0-0731e670308b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:12.067670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15585a2c-b8ec-4b70-8ae0-0731e670308b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:12.068022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15585a2c-b8ec-4b70-8ae0-0731e670308b admin admin] PATH_INFO: `/v3/projects/0ee309dd698d4c34a1188132a35c60fc/users/0ea1adaa89ce47b0925e3b0a5611849e/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:12.090370 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15585a2c-b8ec-4b70-8ae0-0731e670308b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee309dd698d4c34a1188132a35c60fc, user_id=0ea1adaa89ce47b0925e3b0a5611849e, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:12.093697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15585a2c-b8ec-4b70-8ae0-0731e670308b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:12.110725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1178/2352] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:26:12 2022] PUT /identity/v3/projects/0ee309dd698d4c34a1188132a35c60fc/users/0ea1adaa89ce47b0925e3b0a5611849e/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:12.123706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade02c9-1ab5-4fcb-8b9a-55bd7cdb1910 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:12.124073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade02c9-1ab5-4fcb-8b9a-55bd7cdb1910 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:12.124450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade02c9-1ab5-4fcb-8b9a-55bd7cdb1910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:12.164072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2ade02c9-1ab5-4fcb-8b9a-55bd7cdb1910 None None] MFA Rules not processed for user `0ea1adaa89ce47b0925e3b0a5611849e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:12.165600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2ade02c9-1ab5-4fcb-8b9a-55bd7cdb1910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:12.267535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1175/2353] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:26:12.278685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15585a2c-b8ec-4b70-8ae0-0731e670308b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:12.280899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-15585a2c-b8ec-4b70-8ae0-0731e670308b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:12.348329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15585a2c-b8ec-4b70-8ae0-0731e670308b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:12.352960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:12.353586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:12.353863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:12.354798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:12.356671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:12.426186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:12.428025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:12.444459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1179/2354] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:14.110489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade02c9-1ab5-4fcb-8b9a-55bd7cdb1910 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:14.126628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade02c9-1ab5-4fcb-8b9a-55bd7cdb1910 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:14.128984 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a313473-f273-475c-91dd-29a30dbd430f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:14.129636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a313473-f273-475c-91dd-29a30dbd430f service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:14.129940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a313473-f273-475c-91dd-29a30dbd430f service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:14.130216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a313473-f273-475c-91dd-29a30dbd430f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:14.136743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1a313473-f273-475c-91dd-29a30dbd430f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:14.219443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a313473-f273-475c-91dd-29a30dbd430f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:14.220969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a313473-f273-475c-91dd-29a30dbd430f service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:14.233104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1176/2355] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:14.967128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:14.968724 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:14.982636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464ebd7-86dd-497a-bc46-ab021a7f27e3 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:14.989979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27667b-09bd-4e81-ac9b-89711be6081e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:14.990276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27667b-09bd-4e81-ac9b-89711be6081e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:14.990660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27667b-09bd-4e81-ac9b-89711be6081e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:14.990999 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27667b-09bd-4e81-ac9b-89711be6081e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:15.002629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27667b-09bd-4e81-ac9b-89711be6081e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:15.004198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27667b-09bd-4e81-ac9b-89711be6081e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:15.016095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1180/2356] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:19.210976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a313473-f273-475c-91dd-29a30dbd430f service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:19.239260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:19.296663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a313473-f273-475c-91dd-29a30dbd430f service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:19.301469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:19.302103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:19.302379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:19.302649 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] PATH_INFO: `/v3/users/cf681cdbc80e441b888b0214be3b4d90` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:19.303795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf681cdbc80e441b888b0214be3b4d90)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:19.305794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:19.378862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf681cdbc80e441b888b0214be3b4d90'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:19.387214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf681cdbc80e441b888b0214be3b4d90'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:19.387995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf681cdbc80e441b888b0214be3b4d90'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:19.403017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf681cdbc80e441b888b0214be3b4d90'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:19.411207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf681cdbc80e441b888b0214be3b4d90'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:19.420409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1177/2357] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:26:19 2022] DELETE /identity/v3/users/cf681cdbc80e441b888b0214be3b4d90 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:19.425207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32709fcd-7f18-4668-a2eb-c2f85c304d49 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:19.425732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32709fcd-7f18-4668-a2eb-c2f85c304d49 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:19.425836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32709fcd-7f18-4668-a2eb-c2f85c304d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:19.447126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-32709fcd-7f18-4668-a2eb-c2f85c304d49 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:19.451029 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-32709fcd-7f18-4668-a2eb-c2f85c304d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:19.516810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1181/2358] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:26:19.527063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:19.528845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:19.592183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a7973-904e-41c1-845a-9c52ad31953e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:19.595889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef02877-32c9-4268-94f0-773f7bbd50bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:19.596529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef02877-32c9-4268-94f0-773f7bbd50bf service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:19.596816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef02877-32c9-4268-94f0-773f7bbd50bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:19.597092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef02877-32c9-4268-94f0-773f7bbd50bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:19.606780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef02877-32c9-4268-94f0-773f7bbd50bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:19.608432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef02877-32c9-4268-94f0-773f7bbd50bf service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:19.618510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1178/2359] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:19.964540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32709fcd-7f18-4668-a2eb-c2f85c304d49 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:19.966001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-32709fcd-7f18-4668-a2eb-c2f85c304d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:19.974426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:19.982950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32709fcd-7f18-4668-a2eb-c2f85c304d49 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:19.988735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:19.989383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:19.989667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:19.989971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] PATH_INFO: `/v3/projects/ce54331eb3d34946865d569b51800674` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:19.991109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce54331eb3d34946865d569b51800674)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:19.993114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:20.051343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce54331eb3d34946865d569b51800674', 'request_id': 'req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.065744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce54331eb3d34946865d569b51800674', 'request_id': 'req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.073397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce54331eb3d34946865d569b51800674', 'request_id': 'req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.085586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] The token cache is being invalidate because project ce54331eb3d34946865d569b51800674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:26:20.085883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.086543 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.088041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1182/2360] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:26:19 2022] DELETE /identity/v3/projects/ce54331eb3d34946865d569b51800674 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:20.092686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef02877-32c9-4268-94f0-773f7bbd50bf service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:20.105439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef02877-32c9-4268-94f0-773f7bbd50bf service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:20.160680 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:20.174989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:20.175629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:20.175915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:20.176191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] PATH_INFO: `/v3/users/30784403db1743b08851f44a0d75b335` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:20.177386 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30784403db1743b08851f44a0d75b335)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:20.179365 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:20.237555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30784403db1743b08851f44a0d75b335'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.246592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30784403db1743b08851f44a0d75b335'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.247263 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30784403db1743b08851f44a0d75b335'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.263127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30784403db1743b08851f44a0d75b335'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.270844 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30784403db1743b08851f44a0d75b335'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.277590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1179/2361] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:26:20 2022] DELETE /identity/v3/users/30784403db1743b08851f44a0d75b335 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:20.571169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:20.573848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:20.590036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e07d63-6391-4cf0-9bd9-4d32fa2f27fb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:20.656163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:20.656837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:20.657116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:20.657391 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] PATH_INFO: `/v3/projects/f45dcc8ac2664c8ea6d0250d9ef3b9da` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:20.658576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f45dcc8ac2664c8ea6d0250d9ef3b9da)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:20.660601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:20.712092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f45dcc8ac2664c8ea6d0250d9ef3b9da', 'request_id': 'req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.718051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f45dcc8ac2664c8ea6d0250d9ef3b9da', 'request_id': 'req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.726483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f45dcc8ac2664c8ea6d0250d9ef3b9da', 'request_id': 'req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.742939 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] The token cache is being invalidate because project f45dcc8ac2664c8ea6d0250d9ef3b9da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:26:20.743226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.743882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.746486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1183/2362] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:26:20 2022] DELETE /identity/v3/projects/f45dcc8ac2664c8ea6d0250d9ef3b9da => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:20.750610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:20.752128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:20.767440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419f05-7a62-4c59-9f3c-7dcef410e969 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:20.838901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:20.839667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:20.840776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:20.841052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] PATH_INFO: `/v3/users/0ea1adaa89ce47b0925e3b0a5611849e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:20.842370 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea1adaa89ce47b0925e3b0a5611849e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:20.844763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:20.904766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea1adaa89ce47b0925e3b0a5611849e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.912811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea1adaa89ce47b0925e3b0a5611849e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.913566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea1adaa89ce47b0925e3b0a5611849e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.920727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:20.930704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea1adaa89ce47b0925e3b0a5611849e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.942001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22820c63-495e-49e4-adfc-e6a8681522b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea1adaa89ce47b0925e3b0a5611849e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:20.952555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1180/2363] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:26:20 2022] DELETE /identity/v3/users/0ea1adaa89ce47b0925e3b0a5611849e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:21.271298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:21.273265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:21.288794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca44226-c403-4fe5-95c4-16eac9ddfb3e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:21.318289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:21.369714 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:21.370357 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:21.370756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:21.371036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] PATH_INFO: `/v3/projects/0ee309dd698d4c34a1188132a35c60fc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:21.372198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee309dd698d4c34a1188132a35c60fc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:21.374219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:21.433571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee309dd698d4c34a1188132a35c60fc', 'request_id': 'req-09d2b715-dcac-4849-91ac-2f05b00f1063'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:21.439240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee309dd698d4c34a1188132a35c60fc', 'request_id': 'req-09d2b715-dcac-4849-91ac-2f05b00f1063'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:21.447600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee309dd698d4c34a1188132a35c60fc', 'request_id': 'req-09d2b715-dcac-4849-91ac-2f05b00f1063'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:21.460744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] The token cache is being invalidate because project 0ee309dd698d4c34a1188132a35c60fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:26:21.461014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:21.461686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-09d2b715-dcac-4849-91ac-2f05b00f1063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:26:21.463188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1184/2364] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:26:21 2022] DELETE /identity/v3/projects/0ee309dd698d4c34a1188132a35c60fc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:26:26.678747 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c134f-4f0a-4e5e-93c8-1cb732640190 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:26.679207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c134f-4f0a-4e5e-93c8-1cb732640190 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:26.679302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c134f-4f0a-4e5e-93c8-1cb732640190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:26.699329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3d6c134f-4f0a-4e5e-93c8-1cb732640190 None None] MFA Rules not processed for user `af3e7eea0b2840b79404b370382ab44a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:26.700589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3d6c134f-4f0a-4e5e-93c8-1cb732640190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:26.762811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1181/2365] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:29.675838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7b11e-9447-4f5a-b036-97b714176a93 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:29.676455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7b11e-9447-4f5a-b036-97b714176a93 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:29.676579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7b11e-9447-4f5a-b036-97b714176a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:29.703660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-60b7b11e-9447-4f5a-b036-97b714176a93 None None] MFA Rules not processed for user `1be80a6a93454cd8b7128e976308bd4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:26:29.706173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-60b7b11e-9447-4f5a-b036-97b714176a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:29.765255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1185/2366] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:26:36.755627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=31,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:39.757595 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:26:46.723992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c134f-4f0a-4e5e-93c8-1cb732640190 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:26:46.725437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3d6c134f-4f0a-4e5e-93c8-1cb732640190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:46.786142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c134f-4f0a-4e5e-93c8-1cb732640190 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:26:46.789846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:26:46.790446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:26:46.790723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:26:46.790996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:26:46.793070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:26:46.861044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:26:46.862512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:26:46.872350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1182/2367] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:26:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:26:56.867808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=50 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:27:27.230332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb93d4-4484-43c4-892c-cb30a23a38ed None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:27.230896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb93d4-4484-43c4-892c-cb30a23a38ed None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:27.230896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb93d4-4484-43c4-892c-cb30a23a38ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:27.250628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ebdb93d4-4484-43c4-892c-cb30a23a38ed None None] MFA Rules not processed for user `4aaa94a183fb49b79e6383ea783ab76d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:27.251867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ebdb93d4-4484-43c4-892c-cb30a23a38ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:27.319074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1186/2368] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:27:30.086903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:30.089152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:30.198704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26566d4-1c7b-4a6a-9d00-3d09ecadce8d service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:30.205555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:30.206212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:30.206491 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:30.206911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] PATH_INFO: `/v3/users/1be80a6a93454cd8b7128e976308bd4b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:30.208162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be80a6a93454cd8b7128e976308bd4b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:30.210681 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:30.297684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be80a6a93454cd8b7128e976308bd4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.313879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be80a6a93454cd8b7128e976308bd4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.315910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be80a6a93454cd8b7128e976308bd4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.332000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be80a6a93454cd8b7128e976308bd4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.342909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd7fb754-bac5-4797-a0eb-356a095ab33d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be80a6a93454cd8b7128e976308bd4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.350335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1183/2369] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:27:30 2022] DELETE /identity/v3/users/1be80a6a93454cd8b7128e976308bd4b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:27:30.596757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb93d4-4484-43c4-892c-cb30a23a38ed None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:30.610013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb93d4-4484-43c4-892c-cb30a23a38ed None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:30.714392 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:30.715282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:30.715570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:30.715849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] PATH_INFO: `/v3/projects/42b848bb5725477b8832057b56c9cf4d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:30.717175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b848bb5725477b8832057b56c9cf4d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:30.719314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:30.776041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b848bb5725477b8832057b56c9cf4d', 'request_id': 'req-3fdd29e9-dd4e-4368-b756-716af1b5e56e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.782121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b848bb5725477b8832057b56c9cf4d', 'request_id': 'req-3fdd29e9-dd4e-4368-b756-716af1b5e56e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.791321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b848bb5725477b8832057b56c9cf4d', 'request_id': 'req-3fdd29e9-dd4e-4368-b756-716af1b5e56e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.805401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] The token cache is being invalidate because project 42b848bb5725477b8832057b56c9cf4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:27:30.805729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.806501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3fdd29e9-dd4e-4368-b756-716af1b5e56e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:30.809096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1187/2370] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:27:30 2022] DELETE /identity/v3/projects/42b848bb5725477b8832057b56c9cf4d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:27:30.813835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d23aa-761e-40e1-b081-e8e6e76bba7c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:30.814120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d23aa-761e-40e1-b081-e8e6e76bba7c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:30.814392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d23aa-761e-40e1-b081-e8e6e76bba7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:30.836123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d13d23aa-761e-40e1-b081-e8e6e76bba7c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:30.837566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d13d23aa-761e-40e1-b081-e8e6e76bba7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:30.903611 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1184/2371] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:27:30.911642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d48a47-4ded-4bae-8804-de7498aec491 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:30.911923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d48a47-4ded-4bae-8804-de7498aec491 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:30.912231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d48a47-4ded-4bae-8804-de7498aec491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:30.936484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-43d48a47-4ded-4bae-8804-de7498aec491 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:30.938103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-43d48a47-4ded-4bae-8804-de7498aec491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:30.956353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1188/2372] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:27:30.960936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d23aa-761e-40e1-b081-e8e6e76bba7c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:30.970991 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d23aa-761e-40e1-b081-e8e6e76bba7c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:30.975590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37045723-f093-4ad7-bd33-9e55d201b003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:30.976419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37045723-f093-4ad7-bd33-9e55d201b003 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:30.976673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37045723-f093-4ad7-bd33-9e55d201b003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:30.976854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37045723-f093-4ad7-bd33-9e55d201b003 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:30.977514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37045723-f093-4ad7-bd33-9e55d201b003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:27:30.977837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37045723-f093-4ad7-bd33-9e55d201b003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:30.980010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37045723-f093-4ad7-bd33-9e55d201b003 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:30.992593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1185/2373] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:27:30.997636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d48a47-4ded-4bae-8804-de7498aec491 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:31.007180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d48a47-4ded-4bae-8804-de7498aec491 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:31.012249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2172bb8-c398-4058-8dad-c9a534ae6746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:31.013039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2172bb8-c398-4058-8dad-c9a534ae6746 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:31.013322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2172bb8-c398-4058-8dad-c9a534ae6746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:31.013595 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2172bb8-c398-4058-8dad-c9a534ae6746 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:31.014232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2172bb8-c398-4058-8dad-c9a534ae6746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:31.016179 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2172bb8-c398-4058-8dad-c9a534ae6746 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:31.046822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1189/2374] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:27:30 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:27:31.051306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37045723-f093-4ad7-bd33-9e55d201b003 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:31.062323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37045723-f093-4ad7-bd33-9e55d201b003 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:31.066392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:27:31.129541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734c6a-7c80-457b-91ab-e513206ae255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:31.130210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734c6a-7c80-457b-91ab-e513206ae255 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:31.130351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734c6a-7c80-457b-91ab-e513206ae255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:31.130671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734c6a-7c80-457b-91ab-e513206ae255 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:31.131293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7734c6a-7c80-457b-91ab-e513206ae255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:31.133290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7734c6a-7c80-457b-91ab-e513206ae255 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:31.156689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1186/2375] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:27:31 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:27:31.160949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2172bb8-c398-4058-8dad-c9a534ae6746 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:31.172308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2172bb8-c398-4058-8dad-c9a534ae6746 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:31.176982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef43bc2-ace4-4c06-9d23-95e881cb3cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:31.177583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef43bc2-ace4-4c06-9d23-95e881cb3cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:31.177860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef43bc2-ace4-4c06-9d23-95e881cb3cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:31.178170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef43bc2-ace4-4c06-9d23-95e881cb3cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:31.178769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef43bc2-ace4-4c06-9d23-95e881cb3cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:31.180411 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef43bc2-ace4-4c06-9d23-95e881cb3cbd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:31.181824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:27:31.192655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1190/2376] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:27:31.196785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734c6a-7c80-457b-91ab-e513206ae255 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:31.207158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734c6a-7c80-457b-91ab-e513206ae255 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:31.211972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47eb16-0830-4a95-b1f9-296958f8e5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:31.212616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47eb16-0830-4a95-b1f9-296958f8e5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:31.212900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47eb16-0830-4a95-b1f9-296958f8e5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:31.213173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47eb16-0830-4a95-b1f9-296958f8e5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:31.214518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47eb16-0830-4a95-b1f9-296958f8e5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:31.216138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47eb16-0830-4a95-b1f9-296958f8e5d4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:31.227228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1187/2377] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:27:31.232593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef43bc2-ace4-4c06-9d23-95e881cb3cbd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:31.242930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef43bc2-ace4-4c06-9d23-95e881cb3cbd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:31.248351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce05aa-4950-4e39-b454-860896dc07bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:31.249061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce05aa-4950-4e39-b454-860896dc07bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:31.249406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce05aa-4950-4e39-b454-860896dc07bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:31.249748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce05aa-4950-4e39-b454-860896dc07bf admin admin] PATH_INFO: `/v3/projects/3c5c76ac13f74a10b83c33bc25e681bd/users/7c99a103aafb4179b879fb5eb187230a/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:31.266536 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce05aa-4950-4e39-b454-860896dc07bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5c76ac13f74a10b83c33bc25e681bd, user_id=7c99a103aafb4179b879fb5eb187230a, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:31.269278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce05aa-4950-4e39-b454-860896dc07bf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:31.284444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1191/2378] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:27:31 2022] PUT /identity/v3/projects/3c5c76ac13f74a10b83c33bc25e681bd/users/7c99a103aafb4179b879fb5eb187230a/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:27:31.296846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244132-8b22-46a0-9c3b-e02afd00f8b6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:31.297137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244132-8b22-46a0-9c3b-e02afd00f8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:31.297416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244132-8b22-46a0-9c3b-e02afd00f8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:31.328119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f4244132-8b22-46a0-9c3b-e02afd00f8b6 None None] MFA Rules not processed for user `7c99a103aafb4179b879fb5eb187230a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:31.329449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f4244132-8b22-46a0-9c3b-e02afd00f8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:31.410205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1188/2379] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:27:31.421684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce05aa-4950-4e39-b454-860896dc07bf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:31.423469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-04ce05aa-4950-4e39-b454-860896dc07bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:31.513862 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce05aa-4950-4e39-b454-860896dc07bf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:31.518102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0a311-a19f-4698-a0ee-1b5be525bc1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:31.518899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0a311-a19f-4698-a0ee-1b5be525bc1a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:31.519273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0a311-a19f-4698-a0ee-1b5be525bc1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:31.519638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0a311-a19f-4698-a0ee-1b5be525bc1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:31.530341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0a311-a19f-4698-a0ee-1b5be525bc1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:31.532106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0a311-a19f-4698-a0ee-1b5be525bc1a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:31.542632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1192/2380] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:27:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:27:41.407051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:27:41.537601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:27:45.740933 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902a5d1-1f74-4bf7-8490-57869b091845 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:45.741751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902a5d1-1f74-4bf7-8490-57869b091845 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:45.742021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902a5d1-1f74-4bf7-8490-57869b091845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:45.765750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e902a5d1-1f74-4bf7-8490-57869b091845 None None] MFA Rules not processed for user `218f3dd17e6549fc8796eb30e4b48e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:45.767066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e902a5d1-1f74-4bf7-8490-57869b091845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:45.846918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1189/2381] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:27:50.119223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0a311-a19f-4698-a0ee-1b5be525bc1a service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:50.182359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0a311-a19f-4698-a0ee-1b5be525bc1a service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:50.187934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:50.188593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:50.188871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:50.189141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] PATH_INFO: `/v3/users/7c99a103aafb4179b879fb5eb187230a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:50.190287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c99a103aafb4179b879fb5eb187230a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:50.192272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:50.244412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c99a103aafb4179b879fb5eb187230a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.245112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c99a103aafb4179b879fb5eb187230a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.245712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c99a103aafb4179b879fb5eb187230a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.253974 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c99a103aafb4179b879fb5eb187230a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.267801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c99a103aafb4179b879fb5eb187230a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.290771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07896607-304a-48f2-a465-ce0313342610 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:50.291339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07896607-304a-48f2-a465-ce0313342610 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:50.291339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07896607-304a-48f2-a465-ce0313342610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:50.292469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1193/2382] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:27:50 2022] DELETE /identity/v3/users/7c99a103aafb4179b879fb5eb187230a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:27:50.311261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-07896607-304a-48f2-a465-ce0313342610 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:50.312493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-07896607-304a-48f2-a465-ce0313342610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:50.373851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1190/2383] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:27:50.383678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:50.385129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:50.454696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604da864-e4dd-4194-a5fd-b5c291220b62 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:50.458285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4ece0-8f5a-4416-a23d-996f2115233c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:50.458923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4ece0-8f5a-4416-a23d-996f2115233c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:50.459203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4ece0-8f5a-4416-a23d-996f2115233c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:50.459477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4ece0-8f5a-4416-a23d-996f2115233c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:50.469346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4ece0-8f5a-4416-a23d-996f2115233c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:50.470831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4ece0-8f5a-4416-a23d-996f2115233c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:50.481551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1194/2384] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:27:50.772830 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07896607-304a-48f2-a465-ce0313342610 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:50.774280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-07896607-304a-48f2-a465-ce0313342610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:50.787550 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07896607-304a-48f2-a465-ce0313342610 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:50.792718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:50.793897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:50.794268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:50.794450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] PATH_INFO: `/v3/projects/3c5c76ac13f74a10b83c33bc25e681bd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:50.795632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5c76ac13f74a10b83c33bc25e681bd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:50.797892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:50.848856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5c76ac13f74a10b83c33bc25e681bd', 'request_id': 'req-aefbb5a1-f6af-4b1a-864f-b8606c53b798'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.855032 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5c76ac13f74a10b83c33bc25e681bd', 'request_id': 'req-aefbb5a1-f6af-4b1a-864f-b8606c53b798'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.865614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5c76ac13f74a10b83c33bc25e681bd', 'request_id': 'req-aefbb5a1-f6af-4b1a-864f-b8606c53b798'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.880506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] The token cache is being invalidate because project 3c5c76ac13f74a10b83c33bc25e681bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:27:50.880810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.881921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-aefbb5a1-f6af-4b1a-864f-b8606c53b798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:27:50.883674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1191/2385] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:27:50 2022] DELETE /identity/v3/projects/3c5c76ac13f74a10b83c33bc25e681bd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:27:50.890303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac3524-9251-4971-a6f1-26730028cdd8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:50.890661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac3524-9251-4971-a6f1-26730028cdd8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:50.890929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac3524-9251-4971-a6f1-26730028cdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:50.923617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9dac3524-9251-4971-a6f1-26730028cdd8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:50.924687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9dac3524-9251-4971-a6f1-26730028cdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:50.951587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:27:51.018921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1195/2386] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:27:51.027074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaf3d8-e623-4e60-ad71-9425fce32ba9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:51.027423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaf3d8-e623-4e60-ad71-9425fce32ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:51.028210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfaf3d8-e623-4e60-ad71-9425fce32ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:51.055304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9cfaf3d8-e623-4e60-ad71-9425fce32ba9 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:51.056683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9cfaf3d8-e623-4e60-ad71-9425fce32ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:51.072754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1192/2387] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:27:51.077813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac3524-9251-4971-a6f1-26730028cdd8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:51.089367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac3524-9251-4971-a6f1-26730028cdd8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:51.096624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fad16f-240f-4b11-804b-f9770ca980c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:51.097647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fad16f-240f-4b11-804b-f9770ca980c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:51.098151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fad16f-240f-4b11-804b-f9770ca980c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:51.098836 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fad16f-240f-4b11-804b-f9770ca980c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:51.099729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fad16f-240f-4b11-804b-f9770ca980c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:27:51.100264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fad16f-240f-4b11-804b-f9770ca980c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:51.102571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fad16f-240f-4b11-804b-f9770ca980c7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:51.127578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1196/2388] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:27:51.132220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaf3d8-e623-4e60-ad71-9425fce32ba9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:51.143317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfaf3d8-e623-4e60-ad71-9425fce32ba9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:51.148775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ff5c8-62c6-4522-a00c-d427dfc80efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:51.149403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff5c8-62c6-4522-a00c-d427dfc80efd admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:51.149687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff5c8-62c6-4522-a00c-d427dfc80efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:51.149966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff5c8-62c6-4522-a00c-d427dfc80efd admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:51.150631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ff5c8-62c6-4522-a00c-d427dfc80efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:51.153273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ff5c8-62c6-4522-a00c-d427dfc80efd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:51.180822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:27:51.194670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fad16f-240f-4b11-804b-f9770ca980c7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:51.200587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1193/2389] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:27:51 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:27:51.206409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fad16f-240f-4b11-804b-f9770ca980c7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:51.283692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37124e-f70e-43a8-8617-947a858cc1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:51.284218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37124e-f70e-43a8-8617-947a858cc1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:51.284552 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37124e-f70e-43a8-8617-947a858cc1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:51.284824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37124e-f70e-43a8-8617-947a858cc1e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:51.285481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37124e-f70e-43a8-8617-947a858cc1e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:51.287467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37124e-f70e-43a8-8617-947a858cc1e6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:51.319650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1197/2390] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:27:51 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:27:51.324437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ff5c8-62c6-4522-a00c-d427dfc80efd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:51.335842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ff5c8-62c6-4522-a00c-d427dfc80efd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:51.343382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c99bd-04b8-4572-9732-10fccebb885c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:51.344440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c99bd-04b8-4572-9732-10fccebb885c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:51.344954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c99bd-04b8-4572-9732-10fccebb885c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:51.345463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c99bd-04b8-4572-9732-10fccebb885c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:51.346724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c99bd-04b8-4572-9732-10fccebb885c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:51.349406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c99bd-04b8-4572-9732-10fccebb885c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:51.368120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1194/2391] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:27:51.369904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37124e-f70e-43a8-8617-947a858cc1e6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:51.381361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37124e-f70e-43a8-8617-947a858cc1e6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:51.388649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6f261-d6d4-4443-9f35-30db4fb6bfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:51.389264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f261-d6d4-4443-9f35-30db4fb6bfcd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:51.389541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f261-d6d4-4443-9f35-30db4fb6bfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:51.389816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f261-d6d4-4443-9f35-30db4fb6bfcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:51.390448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6f261-d6d4-4443-9f35-30db4fb6bfcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:51.392059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6f261-d6d4-4443-9f35-30db4fb6bfcd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:51.403993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1198/2392] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:27:51.408030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c99bd-04b8-4572-9732-10fccebb885c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:51.419738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c99bd-04b8-4572-9732-10fccebb885c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:51.425636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4ecda-99b1-48e0-aba5-f2d92eb5ab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:51.426323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4ecda-99b1-48e0-aba5-f2d92eb5ab84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:51.426605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4ecda-99b1-48e0-aba5-f2d92eb5ab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:51.426878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4ecda-99b1-48e0-aba5-f2d92eb5ab84 admin admin] PATH_INFO: `/v3/projects/3c3e603ab69848a2b9688f1e536017f3/users/48b7030c933c4e6a92711b35cd6f9685/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:51.443026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4ecda-99b1-48e0-aba5-f2d92eb5ab84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c3e603ab69848a2b9688f1e536017f3, user_id=48b7030c933c4e6a92711b35cd6f9685, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:51.445799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4ecda-99b1-48e0-aba5-f2d92eb5ab84 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:51.461856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1195/2393] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:27:51 2022] PUT /identity/v3/projects/3c3e603ab69848a2b9688f1e536017f3/users/48b7030c933c4e6a92711b35cd6f9685/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:27:51.467861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69feaed-084f-42c7-9200-df0f74c78d45 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:51.468693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69feaed-084f-42c7-9200-df0f74c78d45 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:51.469046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69feaed-084f-42c7-9200-df0f74c78d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:51.492491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f69feaed-084f-42c7-9200-df0f74c78d45 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:51.493739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f69feaed-084f-42c7-9200-df0f74c78d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:51.563759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1199/2394] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:27:51.574847 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4ecda-99b1-48e0-aba5-f2d92eb5ab84 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:51.577257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d6d4ecda-99b1-48e0-aba5-f2d92eb5ab84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:51.638754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4ecda-99b1-48e0-aba5-f2d92eb5ab84 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:51.642616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3816ae-6a6f-4ea4-95f8-c3a67bed54c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:51.643246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3816ae-6a6f-4ea4-95f8-c3a67bed54c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:51.643528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3816ae-6a6f-4ea4-95f8-c3a67bed54c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:51.643804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3816ae-6a6f-4ea4-95f8-c3a67bed54c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:51.653216 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3816ae-6a6f-4ea4-95f8-c3a67bed54c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:51.654676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3816ae-6a6f-4ea4-95f8-c3a67bed54c8 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:51.664636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1196/2395] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:27:56.096329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771f590-5de9-4710-b473-704ee2be0ceb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:56.096903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771f590-5de9-4710-b473-704ee2be0ceb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:56.096903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771f590-5de9-4710-b473-704ee2be0ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:56.135782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c771f590-5de9-4710-b473-704ee2be0ceb None None] MFA Rules not processed for user `48b7030c933c4e6a92711b35cd6f9685`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:27:56.137860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c771f590-5de9-4710-b473-704ee2be0ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:27:56.158126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:27:56.231302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1200/2396] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:27:56.233521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3816ae-6a6f-4ea4-95f8-c3a67bed54c8 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:56.251827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3816ae-6a6f-4ea4-95f8-c3a67bed54c8 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:56.256944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ad073-9d75-4800-bd42-dfa4fb7a8f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:56.257545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ad073-9d75-4800-bd42-dfa4fb7a8f62 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:56.257823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ad073-9d75-4800-bd42-dfa4fb7a8f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:56.258095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ad073-9d75-4800-bd42-dfa4fb7a8f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:56.258723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ad073-9d75-4800-bd42-dfa4fb7a8f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:56.260682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ad073-9d75-4800-bd42-dfa4fb7a8f62 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:56.290340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1197/2397] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:27:56 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:27:56.295332 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771f590-5de9-4710-b473-704ee2be0ceb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:56.306493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771f590-5de9-4710-b473-704ee2be0ceb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:56.363563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14b2cd-918f-4a17-b758-8ec5392073f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:56.364186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14b2cd-918f-4a17-b758-8ec5392073f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:56.364490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14b2cd-918f-4a17-b758-8ec5392073f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:56.364764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14b2cd-918f-4a17-b758-8ec5392073f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:56.365395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14b2cd-918f-4a17-b758-8ec5392073f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:56.367335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14b2cd-918f-4a17-b758-8ec5392073f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:56.391468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1201/2398] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:27:56 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:27:56.392651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ad073-9d75-4800-bd42-dfa4fb7a8f62 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:56.403703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ad073-9d75-4800-bd42-dfa4fb7a8f62 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:56.408869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42b4b2-a1fb-42b7-906a-0340106361a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:56.409472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b4b2-a1fb-42b7-906a-0340106361a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:56.409755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b4b2-a1fb-42b7-906a-0340106361a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:56.410030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b4b2-a1fb-42b7-906a-0340106361a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:56.410705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42b4b2-a1fb-42b7-906a-0340106361a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:56.412349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42b4b2-a1fb-42b7-906a-0340106361a1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:56.424902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1198/2399] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:27:56.429261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14b2cd-918f-4a17-b758-8ec5392073f8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:56.440289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14b2cd-918f-4a17-b758-8ec5392073f8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:56.445315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8709c5-8ae7-4fdd-a763-d55bf83df99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:56.445984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8709c5-8ae7-4fdd-a763-d55bf83df99e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:56.446267 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8709c5-8ae7-4fdd-a763-d55bf83df99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:56.446544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8709c5-8ae7-4fdd-a763-d55bf83df99e admin admin] PATH_INFO: `/v3/projects/036bcceabe0f4077a91d5c5d4bdd1b11/users/7e5eb15ec8514b4b97cee01f9d360cfe/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:56.463379 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8709c5-8ae7-4fdd-a763-d55bf83df99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=036bcceabe0f4077a91d5c5d4bdd1b11, user_id=7e5eb15ec8514b4b97cee01f9d360cfe, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:56.466741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8709c5-8ae7-4fdd-a763-d55bf83df99e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:56.481513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1202/2400] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:27:56 2022] PUT /identity/v3/projects/036bcceabe0f4077a91d5c5d4bdd1b11/users/7e5eb15ec8514b4b97cee01f9d360cfe/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:27:56.487036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42b4b2-a1fb-42b7-906a-0340106361a1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:56.496089 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42b4b2-a1fb-42b7-906a-0340106361a1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:56.553738 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e24cd-8d05-4879-9ca9-e48df761f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:56.554337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e24cd-8d05-4879-9ca9-e48df761f772 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:56.554616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e24cd-8d05-4879-9ca9-e48df761f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:56.554890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e24cd-8d05-4879-9ca9-e48df761f772 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:56.555533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e24cd-8d05-4879-9ca9-e48df761f772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:56.557176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0e24cd-8d05-4879-9ca9-e48df761f772 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:56.567435 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1199/2401] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:27:56.571526 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8709c5-8ae7-4fdd-a763-d55bf83df99e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:27:56.582947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8709c5-8ae7-4fdd-a763-d55bf83df99e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:27:56.587446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376642a-3192-4c17-b80e-77d347b8d535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:27:56.588087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376642a-3192-4c17-b80e-77d347b8d535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:27:56.588365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376642a-3192-4c17-b80e-77d347b8d535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:27:56.588806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376642a-3192-4c17-b80e-77d347b8d535 admin admin] PATH_INFO: `/v3/projects/036bcceabe0f4077a91d5c5d4bdd1b11/users/7e5eb15ec8514b4b97cee01f9d360cfe/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:27:56.591261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376642a-3192-4c17-b80e-77d347b8d535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036bcceabe0f4077a91d5c5d4bdd1b11, user_id=7e5eb15ec8514b4b97cee01f9d360cfe, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:27:56.594494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376642a-3192-4c17-b80e-77d347b8d535 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:27:56.605082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1203/2402] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:27:56 2022] PUT /identity/v3/projects/036bcceabe0f4077a91d5c5d4bdd1b11/users/7e5eb15ec8514b4b97cee01f9d360cfe/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:28:01.565879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e24cd-8d05-4879-9ca9-e48df761f772 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:01.568638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9a0e24cd-8d05-4879-9ca9-e48df761f772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:01.614810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:28:01.653865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e24cd-8d05-4879-9ca9-e48df761f772 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:01.763459 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:01.764200 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:01.764580 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:01.764921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:01.766955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:01.853675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:01.855168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:01.870854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1200/2403] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:02.583178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bfb4c-5575-453b-a862-e53cdef40990 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:02.583720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bfb4c-5575-453b-a862-e53cdef40990 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:02.583895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bfb4c-5575-453b-a862-e53cdef40990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:02.619219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8f3bfb4c-5575-453b-a862-e53cdef40990 None None] MFA Rules not processed for user `7e5eb15ec8514b4b97cee01f9d360cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:28:02.620624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8f3bfb4c-5575-453b-a862-e53cdef40990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:02.702881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:28:02.729723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1204/2404] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:28:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:28:02.741770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:02.743339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:02.822994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ca814-4262-42dd-b4e1-e9af354d7c41 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:02.827982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5a869-a812-4453-8a7a-83deb90e1172 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0dfcbad22348fdaea32d64557e68a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:02.828627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5a869-a812-4453-8a7a-83deb90e1172 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:02.829061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5a869-a812-4453-8a7a-83deb90e1172 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:02.829339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5a869-a812-4453-8a7a-83deb90e1172 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:02.887852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:28:02.907817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5a869-a812-4453-8a7a-83deb90e1172 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:02.909605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5a869-a812-4453-8a7a-83deb90e1172 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:02.921302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1201/2405] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:03.090325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cbbcd-0e30-44b5-ae79-996ebd4f3668 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:03.090325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cbbcd-0e30-44b5-ae79-996ebd4f3668 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:03.091060 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cbbcd-0e30-44b5-ae79-996ebd4f3668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:03.147713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aa9cbbcd-0e30-44b5-ae79-996ebd4f3668 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:28:03.148782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aa9cbbcd-0e30-44b5-ae79-996ebd4f3668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:03.252688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1205/2406] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:28:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:28:03.301990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5a869-a812-4453-8a7a-83deb90e1172 service glance] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:03.303698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0ac5a869-a812-4453-8a7a-83deb90e1172 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:03.398910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5a869-a812-4453-8a7a-83deb90e1172 service glance] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:03.409656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3b60-edd8-4df0-b73d-6e3fa6f2c104 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:03.411355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3b60-edd8-4df0-b73d-6e3fa6f2c104 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:03.411355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3b60-edd8-4df0-b73d-6e3fa6f2c104 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:03.412060 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3b60-edd8-4df0-b73d-6e3fa6f2c104 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:03.423623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd3b60-edd8-4df0-b73d-6e3fa6f2c104 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:03.429168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd3b60-edd8-4df0-b73d-6e3fa6f2c104 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:03.443633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1202/2407] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:28:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:28:06.174718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fc222-e723-4559-9b70-2e76a7af677d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:06.176265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fc222-e723-4559-9b70-2e76a7af677d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:06.177065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fc222-e723-4559-9b70-2e76a7af677d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:06.207503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-166fc222-e723-4559-9b70-2e76a7af677d None None] MFA Rules not processed for user `dc2bbdceb12641e6a0595e68f7cefd63`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:28:06.209032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-166fc222-e723-4559-9b70-2e76a7af677d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:06.290525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1206/2408] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:28:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:28:06.333328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3b60-edd8-4df0-b73d-6e3fa6f2c104 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:06.346148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3b60-edd8-4df0-b73d-6e3fa6f2c104 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:06.349876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d53a4-0911-4960-9e8b-5d06df90e8a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0dfcbad22348fdaea32d64557e68a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:06.350545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d53a4-0911-4960-9e8b-5d06df90e8a5 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:06.350887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d53a4-0911-4960-9e8b-5d06df90e8a5 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:06.351222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d53a4-0911-4960-9e8b-5d06df90e8a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:06.353086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a60d53a4-0911-4960-9e8b-5d06df90e8a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:06.368881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d53a4-0911-4960-9e8b-5d06df90e8a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:06.370461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d53a4-0911-4960-9e8b-5d06df90e8a5 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:06.386337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1203/2409] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:11.834468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fc222-e723-4559-9b70-2e76a7af677d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:11.836086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-166fc222-e723-4559-9b70-2e76a7af677d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:11.855356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fc222-e723-4559-9b70-2e76a7af677d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:11.862763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3caaed-2069-4bf1-8981-4a689915ed8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:11.863454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3caaed-2069-4bf1-8981-4a689915ed8e service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:11.863728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3caaed-2069-4bf1-8981-4a689915ed8e service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:11.863995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3caaed-2069-4bf1-8981-4a689915ed8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:11.866193 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3f3caaed-2069-4bf1-8981-4a689915ed8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:11.878478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3caaed-2069-4bf1-8981-4a689915ed8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:11.880102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3caaed-2069-4bf1-8981-4a689915ed8e service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:11.891325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1207/2410] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:13.432647 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90182bc3-daa4-4386-9416-dd4d2e3765bb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:13.433168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90182bc3-daa4-4386-9416-dd4d2e3765bb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:13.433168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90182bc3-daa4-4386-9416-dd4d2e3765bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:13.454584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-90182bc3-daa4-4386-9416-dd4d2e3765bb None None] MFA Rules not processed for user `48b7030c933c4e6a92711b35cd6f9685`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:28:13.455795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-90182bc3-daa4-4386-9416-dd4d2e3765bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:13.462877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1204/2411] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:28:15.413741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8dcc9-8011-4a69-b18a-4eea25521758 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:15.414255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8dcc9-8011-4a69-b18a-4eea25521758 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:15.414378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8dcc9-8011-4a69-b18a-4eea25521758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:15.433541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2fa8dcc9-8011-4a69-b18a-4eea25521758 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:28:15.435098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2fa8dcc9-8011-4a69-b18a-4eea25521758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:15.443766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1208/2412] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:28:15.456787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90182bc3-daa4-4386-9416-dd4d2e3765bb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:15.469441 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90182bc3-daa4-4386-9416-dd4d2e3765bb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:15.474950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44dc38e-df1c-49ae-9e9d-d4f0532a27b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:15.475556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dc38e-df1c-49ae-9e9d-d4f0532a27b2 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:15.475981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dc38e-df1c-49ae-9e9d-d4f0532a27b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:15.476260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dc38e-df1c-49ae-9e9d-d4f0532a27b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:15.484676 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44dc38e-df1c-49ae-9e9d-d4f0532a27b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:15.486340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44dc38e-df1c-49ae-9e9d-d4f0532a27b2 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:15.496330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1205/2413] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:28:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:28:23.027288 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8dcc9-8011-4a69-b18a-4eea25521758 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:23.029042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2fa8dcc9-8011-4a69-b18a-4eea25521758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:23.049692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8dcc9-8011-4a69-b18a-4eea25521758 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:23.058574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd373dad-b9ae-4450-a486-b640468884e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:23.059204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd373dad-b9ae-4450-a486-b640468884e6 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:23.059477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd373dad-b9ae-4450-a486-b640468884e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:23.059745 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd373dad-b9ae-4450-a486-b640468884e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:23.061788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cd373dad-b9ae-4450-a486-b640468884e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:23.130870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd373dad-b9ae-4450-a486-b640468884e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:23.132561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd373dad-b9ae-4450-a486-b640468884e6 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:23.143550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1209/2414] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:25.492967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=55 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:28:29.178500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44dc38e-df1c-49ae-9e9d-d4f0532a27b2 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:29.181072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d44dc38e-df1c-49ae-9e9d-d4f0532a27b2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:29.258499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44dc38e-df1c-49ae-9e9d-d4f0532a27b2 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:29.262978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5633e4-8269-4f11-8fe7-7380ff3aad7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:29.264269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5633e4-8269-4f11-8fe7-7380ff3aad7b service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:29.264684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5633e4-8269-4f11-8fe7-7380ff3aad7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:29.265330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5633e4-8269-4f11-8fe7-7380ff3aad7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:29.266961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6a5633e4-8269-4f11-8fe7-7380ff3aad7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:29.346517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5633e4-8269-4f11-8fe7-7380ff3aad7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:29.348634 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5633e4-8269-4f11-8fe7-7380ff3aad7b service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:29.360653 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1206/2415] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:33.139554 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:28:38.708376 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd373dad-b9ae-4450-a486-b640468884e6 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:38.720031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd373dad-b9ae-4450-a486-b640468884e6 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:38.725782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af8dfe-880a-4e98-a659-4e1371b52a1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:38.726386 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af8dfe-880a-4e98-a659-4e1371b52a1f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:38.726661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af8dfe-880a-4e98-a659-4e1371b52a1f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:38.726931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af8dfe-880a-4e98-a659-4e1371b52a1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:38.728681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35af8dfe-880a-4e98-a659-4e1371b52a1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:38.742096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af8dfe-880a-4e98-a659-4e1371b52a1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:38.743684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af8dfe-880a-4e98-a659-4e1371b52a1f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:38.755596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1210/2416] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:39.354625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=48 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:28:45.036513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccaafa-a532-4ce1-a35c-c4531a181934 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:45.037122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccaafa-a532-4ce1-a35c-c4531a181934 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:45.037122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccaafa-a532-4ce1-a35c-c4531a181934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:45.056420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b8ccaafa-a532-4ce1-a35c-c4531a181934 None None] MFA Rules not processed for user `4aaa94a183fb49b79e6383ea783ab76d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:28:45.057618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b8ccaafa-a532-4ce1-a35c-c4531a181934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:45.064490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1207/2417] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:28:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:28:45.414023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af8dfe-880a-4e98-a659-4e1371b52a1f service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:45.423933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af8dfe-880a-4e98-a659-4e1371b52a1f service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:45.427722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:45.428482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:45.428761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:45.429034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:45.430849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:45.441231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:45.442600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:45.450909 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1211/2418] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:28:45.573780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccaafa-a532-4ce1-a35c-c4531a181934 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:45.584605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccaafa-a532-4ce1-a35c-c4531a181934 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:45.588324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3db2f-156c-4a3b-9bbc-e493c222cdfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:45.589022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3db2f-156c-4a3b-9bbc-e493c222cdfd service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:45.591188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3db2f-156c-4a3b-9bbc-e493c222cdfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:45.591188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3db2f-156c-4a3b-9bbc-e493c222cdfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:45.599766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3db2f-156c-4a3b-9bbc-e493c222cdfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:45.601387 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3db2f-156c-4a3b-9bbc-e493c222cdfd service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:45.611963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1208/2419] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:45.861010 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:45.862437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:45.962329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e23d0-627b-468b-814e-da76308dc245 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:45.969316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9e320-0b3d-499b-90a1-09e3dd3b7ed3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:45.970360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9e320-0b3d-499b-90a1-09e3dd3b7ed3 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:45.970549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9e320-0b3d-499b-90a1-09e3dd3b7ed3 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:45.970828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9e320-0b3d-499b-90a1-09e3dd3b7ed3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:45.972913 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-32a9e320-0b3d-499b-90a1-09e3dd3b7ed3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:45.996658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9e320-0b3d-499b-90a1-09e3dd3b7ed3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:45.998467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9e320-0b3d-499b-90a1-09e3dd3b7ed3 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:46.011203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1212/2420] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:48.313584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3db2f-156c-4a3b-9bbc-e493c222cdfd service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:48.323526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3db2f-156c-4a3b-9bbc-e493c222cdfd service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:48.336656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef163430-bf3a-4bdf-a0aa-d0e1478d8db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:48.338119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef163430-bf3a-4bdf-a0aa-d0e1478d8db5 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:48.338802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef163430-bf3a-4bdf-a0aa-d0e1478d8db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:48.339377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef163430-bf3a-4bdf-a0aa-d0e1478d8db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:48.342963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ef163430-bf3a-4bdf-a0aa-d0e1478d8db5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:48.367298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef163430-bf3a-4bdf-a0aa-d0e1478d8db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:48.371886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef163430-bf3a-4bdf-a0aa-d0e1478d8db5 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:48.397881 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1209/2421] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:50.373483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9e320-0b3d-499b-90a1-09e3dd3b7ed3 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:50.390151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9e320-0b3d-499b-90a1-09e3dd3b7ed3 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:50.398364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab334e4-4e8c-44ac-a2e6-50f2c5975068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:50.399315 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab334e4-4e8c-44ac-a2e6-50f2c5975068 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:50.399804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab334e4-4e8c-44ac-a2e6-50f2c5975068 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:50.400146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab334e4-4e8c-44ac-a2e6-50f2c5975068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:50.402056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ab334e4-4e8c-44ac-a2e6-50f2c5975068 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:50.414985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab334e4-4e8c-44ac-a2e6-50f2c5975068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:50.417032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab334e4-4e8c-44ac-a2e6-50f2c5975068 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:50.432191 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1213/2422] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:55.446080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef163430-bf3a-4bdf-a0aa-d0e1478d8db5 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:55.454762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef163430-bf3a-4bdf-a0aa-d0e1478d8db5 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:55.458355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:55.458948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:55.459243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:55.459492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:55.461236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:55.527827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:55.529321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:55.539879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1210/2423] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:58.677976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab334e4-4e8c-44ac-a2e6-50f2c5975068 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:58.699062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab334e4-4e8c-44ac-a2e6-50f2c5975068 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:58.706674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575221c-68bc-4873-9812-c3ea9f2b626c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:58.707669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575221c-68bc-4873-9812-c3ea9f2b626c service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:58.708136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575221c-68bc-4873-9812-c3ea9f2b626c service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:58.708493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575221c-68bc-4873-9812-c3ea9f2b626c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:58.710773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f575221c-68bc-4873-9812-c3ea9f2b626c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:58.733727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575221c-68bc-4873-9812-c3ea9f2b626c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:58.736562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575221c-68bc-4873-9812-c3ea9f2b626c service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:58.738408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:58.740092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:58.757085 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223b0b2-d3bc-44e3-9f14-d6abdd1dd869 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:58.757666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1214/2424] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:28:58.762673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:58.763400 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:58.763748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:58.764248 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] PATH_INFO: `/v3/users/218f3dd17e6549fc8796eb30e4b48e24` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:58.766249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218f3dd17e6549fc8796eb30e4b48e24)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:58.768584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:58.823773 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218f3dd17e6549fc8796eb30e4b48e24'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:58.831500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218f3dd17e6549fc8796eb30e4b48e24'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:58.832109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218f3dd17e6549fc8796eb30e4b48e24'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:58.845277 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218f3dd17e6549fc8796eb30e4b48e24'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:58.852352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7a65d39a-9e14-4372-b936-b737bb4c8ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218f3dd17e6549fc8796eb30e4b48e24'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:58.858261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1211/2425] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:28:58 2022] DELETE /identity/v3/users/218f3dd17e6549fc8796eb30e4b48e24 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:28:59.116667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575221c-68bc-4873-9812-c3ea9f2b626c service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:59.137840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575221c-68bc-4873-9812-c3ea9f2b626c service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:59.197025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:59.197648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.197926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.198201 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] PATH_INFO: `/v3/projects/dcca6c7d30114cd0936eb30381efad7d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.199311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcca6c7d30114cd0936eb30381efad7d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:59.201378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:59.240998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:28:59.249157 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcca6c7d30114cd0936eb30381efad7d', 'request_id': 'req-670673fd-041e-42b9-bbac-dca5568641bc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:59.255160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcca6c7d30114cd0936eb30381efad7d', 'request_id': 'req-670673fd-041e-42b9-bbac-dca5568641bc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:59.261656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcca6c7d30114cd0936eb30381efad7d', 'request_id': 'req-670673fd-041e-42b9-bbac-dca5568641bc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:59.272580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] The token cache is being invalidate because project dcca6c7d30114cd0936eb30381efad7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:28:59.272870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:59.273494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-670673fd-041e-42b9-bbac-dca5568641bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:28:59.274961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1215/2426] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:28:59 2022] DELETE /identity/v3/projects/dcca6c7d30114cd0936eb30381efad7d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:28:59.280005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c3eb7-469e-4b72-99d5-109041c5d657 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.280302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c3eb7-469e-4b72-99d5-109041c5d657 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.280613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c3eb7-469e-4b72-99d5-109041c5d657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.300950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ac2c3eb7-469e-4b72-99d5-109041c5d657 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:28:59.302149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ac2c3eb7-469e-4b72-99d5-109041c5d657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:59.331388 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:28:59.376552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1212/2427] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:28:59.383126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ad58e-7a53-45ae-80ef-41e9cfe3f8ca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.383579 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ad58e-7a53-45ae-80ef-41e9cfe3f8ca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.383891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ad58e-7a53-45ae-80ef-41e9cfe3f8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.405653 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-378ad58e-7a53-45ae-80ef-41e9cfe3f8ca None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:28:59.407493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-378ad58e-7a53-45ae-80ef-41e9cfe3f8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:59.415759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1216/2428] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:28:59.419851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c3eb7-469e-4b72-99d5-109041c5d657 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:59.430788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c3eb7-469e-4b72-99d5-109041c5d657 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:59.435519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca36a79-e9b3-40cc-8f7a-dedef3780b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:59.436142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36a79-e9b3-40cc-8f7a-dedef3780b4d admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.436446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36a79-e9b3-40cc-8f7a-dedef3780b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.436719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36a79-e9b3-40cc-8f7a-dedef3780b4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.437385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36a79-e9b3-40cc-8f7a-dedef3780b4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:28:59.437727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36a79-e9b3-40cc-8f7a-dedef3780b4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:59.439446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36a79-e9b3-40cc-8f7a-dedef3780b4d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:59.451139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1213/2429] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:28:59.455589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ad58e-7a53-45ae-80ef-41e9cfe3f8ca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:59.465262 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ad58e-7a53-45ae-80ef-41e9cfe3f8ca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:59.470663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6205-a867-4e72-b1ad-da42ff82fc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:59.471271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6205-a867-4e72-b1ad-da42ff82fc22 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.471549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6205-a867-4e72-b1ad-da42ff82fc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.471820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6205-a867-4e72-b1ad-da42ff82fc22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.472478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc6205-a867-4e72-b1ad-da42ff82fc22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:59.474400 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc6205-a867-4e72-b1ad-da42ff82fc22 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:59.505715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1217/2430] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:28:59 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:28:59.506852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca36a79-e9b3-40cc-8f7a-dedef3780b4d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:59.517458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca36a79-e9b3-40cc-8f7a-dedef3780b4d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:59.575721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c4aa-fe97-4d09-b94c-6b5c561d3df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:59.576314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c4aa-fe97-4d09-b94c-6b5c561d3df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.576620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c4aa-fe97-4d09-b94c-6b5c561d3df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.576892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c4aa-fe97-4d09-b94c-6b5c561d3df0 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.577505 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79c4aa-fe97-4d09-b94c-6b5c561d3df0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:59.579932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79c4aa-fe97-4d09-b94c-6b5c561d3df0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:59.600687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1214/2431] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:28:59 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:28:59.604797 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6205-a867-4e72-b1ad-da42ff82fc22 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:59.615301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6205-a867-4e72-b1ad-da42ff82fc22 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:59.619989 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78be3e-d5fd-4aba-a858-a346543d1a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:59.620633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78be3e-d5fd-4aba-a858-a346543d1a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.620911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78be3e-d5fd-4aba-a858-a346543d1a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.621206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78be3e-d5fd-4aba-a858-a346543d1a6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.621848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78be3e-d5fd-4aba-a858-a346543d1a6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:59.623489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78be3e-d5fd-4aba-a858-a346543d1a6f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:59.638422 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1218/2432] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:28:59.642934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c4aa-fe97-4d09-b94c-6b5c561d3df0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:59.652991 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c4aa-fe97-4d09-b94c-6b5c561d3df0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:59.658219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82bc99-3934-40c2-bebf-52b62cebc621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:59.658809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82bc99-3934-40c2-bebf-52b62cebc621 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.659088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82bc99-3934-40c2-bebf-52b62cebc621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.659360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82bc99-3934-40c2-bebf-52b62cebc621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.660001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82bc99-3934-40c2-bebf-52b62cebc621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:59.661641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82bc99-3934-40c2-bebf-52b62cebc621 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:59.672202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1215/2433] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:28:59.676823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78be3e-d5fd-4aba-a858-a346543d1a6f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:59.686920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78be3e-d5fd-4aba-a858-a346543d1a6f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:59.691516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cd9de-b951-4c46-85c2-4ca655aac6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:59.692148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cd9de-b951-4c46-85c2-4ca655aac6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.692452 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cd9de-b951-4c46-85c2-4ca655aac6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.692724 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cd9de-b951-4c46-85c2-4ca655aac6e4 admin admin] PATH_INFO: `/v3/projects/981bbb7282bd40d0b5335280d5041fa5/users/821d38464cd3491e81e94d809930d076/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.709042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cd9de-b951-4c46-85c2-4ca655aac6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=981bbb7282bd40d0b5335280d5041fa5, user_id=821d38464cd3491e81e94d809930d076, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:59.711710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cd9de-b951-4c46-85c2-4ca655aac6e4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:59.725196 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1219/2434] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:28:59 2022] PUT /identity/v3/projects/981bbb7282bd40d0b5335280d5041fa5/users/821d38464cd3491e81e94d809930d076/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:28:59.730194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17715c1-6139-40e9-8c4b-f4f5c225b728 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.730484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17715c1-6139-40e9-8c4b-f4f5c225b728 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.730757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17715c1-6139-40e9-8c4b-f4f5c225b728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.749226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f17715c1-6139-40e9-8c4b-f4f5c225b728 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:28:59.750418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f17715c1-6139-40e9-8c4b-f4f5c225b728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:59.808445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1216/2435] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:28:59.818394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cd9de-b951-4c46-85c2-4ca655aac6e4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:28:59.820110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1b8cd9de-b951-4c46-85c2-4ca655aac6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:28:59.876455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cd9de-b951-4c46-85c2-4ca655aac6e4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:28:59.880054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a7231-6c91-4bec-b04d-b5def3e8adaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:28:59.880685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a7231-6c91-4bec-b04d-b5def3e8adaa service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:28:59.880969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a7231-6c91-4bec-b04d-b5def3e8adaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:28:59.881244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a7231-6c91-4bec-b04d-b5def3e8adaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:28:59.890197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a7231-6c91-4bec-b04d-b5def3e8adaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:28:59.892177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a7231-6c91-4bec-b04d-b5def3e8adaa service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:28:59.901529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1220/2436] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:00.131556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17715c1-6139-40e9-8c4b-f4f5c225b728 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:00.133147 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f17715c1-6139-40e9-8c4b-f4f5c225b728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:00.189004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:00.225533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17715c1-6139-40e9-8c4b-f4f5c225b728 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:00.231023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:00.231632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:00.231917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:00.232187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:00.237423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:00.251956 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:00.253425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:00.267052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1217/2437] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:05.002429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7ebf8-56b5-4d23-bdac-abe6f2311d73 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:05.003050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7ebf8-56b5-4d23-bdac-abe6f2311d73 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:05.003050 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7ebf8-56b5-4d23-bdac-abe6f2311d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:05.035418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0fc7ebf8-56b5-4d23-bdac-abe6f2311d73 None None] MFA Rules not processed for user `821d38464cd3491e81e94d809930d076`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:05.037043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0fc7ebf8-56b5-4d23-bdac-abe6f2311d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:05.113306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:05.140899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1221/2438] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:05.149515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:05.152268 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:05.170736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6b843-a468-4d3d-a7c6-de69a7e36769 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:05.178180 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea2673-64df-44b3-bbe3-c973446acc74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:05.179686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea2673-64df-44b3-bbe3-c973446acc74 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:05.180178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea2673-64df-44b3-bbe3-c973446acc74 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:05.180953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea2673-64df-44b3-bbe3-c973446acc74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:05.195241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea2673-64df-44b3-bbe3-c973446acc74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:05.197692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea2673-64df-44b3-bbe3-c973446acc74 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:05.212185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1218/2439] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:29:07.068812 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a78199-2a75-471a-8e90-0e4d7e32b071 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:07.069688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a78199-2a75-471a-8e90-0e4d7e32b071 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:07.070171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a78199-2a75-471a-8e90-0e4d7e32b071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:07.092940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-57a78199-2a75-471a-8e90-0e4d7e32b071 None None] MFA Rules not processed for user `af3e7eea0b2840b79404b370382ab44a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:07.095032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-57a78199-2a75-471a-8e90-0e4d7e32b071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:07.172650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1222/2440] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:07.773669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559d2a3-05a4-49bb-bdd0-823a1a8d33c5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:07.774221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559d2a3-05a4-49bb-bdd0-823a1a8d33c5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:07.774221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559d2a3-05a4-49bb-bdd0-823a1a8d33c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:07.794176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0559d2a3-05a4-49bb-bdd0-823a1a8d33c5 None None] MFA Rules not processed for user `821d38464cd3491e81e94d809930d076`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:07.795672 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0559d2a3-05a4-49bb-bdd0-823a1a8d33c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:07.803357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1219/2441] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:08.212585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a78199-2a75-471a-8e90-0e4d7e32b071 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:08.230757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a78199-2a75-471a-8e90-0e4d7e32b071 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:08.238988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:08.240724 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:08.241502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:08.241990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:08.245004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:08.269652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:08.272582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:08.283002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1223/2442] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:29:17.788675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=25 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:18.278353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=33 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:44.412675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559d2a3-05a4-49bb-bdd0-823a1a8d33c5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:44.422939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559d2a3-05a4-49bb-bdd0-823a1a8d33c5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:44.426762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:44.427345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:44.427622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:44.427891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:44.429635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:44.441235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:44.442663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:44.452128 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1220/2443] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:47.788428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:47.789910 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:47.863316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:47.866570 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:47.879456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6b332-50e1-4545-8861-40b0eb764744 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:47.884183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:47.884845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:47.885122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:47.885395 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] PATH_INFO: `/v3/users/dc2bbdceb12641e6a0595e68f7cefd63` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:47.886529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2bbdceb12641e6a0595e68f7cefd63)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:47.888516 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:47.896223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42a236-dc72-4795-8f68-6d03b6312d50 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:47.901465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5e592-759d-428e-abd3-7baa45e0dabd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:47.902148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5e592-759d-428e-abd3-7baa45e0dabd service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:47.902486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5e592-759d-428e-abd3-7baa45e0dabd service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:47.902826 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5e592-759d-428e-abd3-7baa45e0dabd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:47.904646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a6f5e592-759d-428e-abd3-7baa45e0dabd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:47.917655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5e592-759d-428e-abd3-7baa45e0dabd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:47.919440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5e592-759d-428e-abd3-7baa45e0dabd service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:47.930633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1224/2444] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:47.939286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2bbdceb12641e6a0595e68f7cefd63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:47.946324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2bbdceb12641e6a0595e68f7cefd63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:47.947020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2bbdceb12641e6a0595e68f7cefd63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:47.960987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2bbdceb12641e6a0595e68f7cefd63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:47.970119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-52e689ab-2786-40c4-852f-84a15ca6cad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2bbdceb12641e6a0595e68f7cefd63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:47.976105 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1221/2445] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:29:47 2022] DELETE /identity/v3/users/dc2bbdceb12641e6a0595e68f7cefd63 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:29:48.144675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5e592-759d-428e-abd3-7baa45e0dabd service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:48.155319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5e592-759d-428e-abd3-7baa45e0dabd service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:48.213877 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:48.214504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.214783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.215054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] PATH_INFO: `/v3/projects/8ede055158e24b7c9a935b833e33b89a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.216155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ede055158e24b7c9a935b833e33b89a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:48.218115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:48.263934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ede055158e24b7c9a935b833e33b89a', 'request_id': 'req-ad9c8de9-b919-4c77-a136-605305e7ee38'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:48.269824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ede055158e24b7c9a935b833e33b89a', 'request_id': 'req-ad9c8de9-b919-4c77-a136-605305e7ee38'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:48.276478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ede055158e24b7c9a935b833e33b89a', 'request_id': 'req-ad9c8de9-b919-4c77-a136-605305e7ee38'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:48.287720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] The token cache is being invalidate because project 8ede055158e24b7c9a935b833e33b89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:29:48.288013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:48.288636 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad9c8de9-b919-4c77-a136-605305e7ee38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:48.290102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1225/2446] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:29:48 2022] DELETE /identity/v3/projects/8ede055158e24b7c9a935b833e33b89a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:48.294952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a525-16c1-4db0-9d9b-de1c91686328 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.295243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a525-16c1-4db0-9d9b-de1c91686328 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.295518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a525-16c1-4db0-9d9b-de1c91686328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.316024 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-db37a525-16c1-4db0-9d9b-de1c91686328 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:48.317273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-db37a525-16c1-4db0-9d9b-de1c91686328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:48.376208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1222/2447] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:29:48.384226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06927e4a-5697-451f-b996-9be7933b2009 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.384564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06927e4a-5697-451f-b996-9be7933b2009 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.384844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06927e4a-5697-451f-b996-9be7933b2009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.404623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-06927e4a-5697-451f-b996-9be7933b2009 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:48.407493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-06927e4a-5697-451f-b996-9be7933b2009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:48.415596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1226/2448] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:48.419811 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a525-16c1-4db0-9d9b-de1c91686328 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:48.430247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a525-16c1-4db0-9d9b-de1c91686328 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:48.434780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae842290-bfd5-4740-b363-6ba4dbdb9694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:48.435457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae842290-bfd5-4740-b363-6ba4dbdb9694 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.435744 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae842290-bfd5-4740-b363-6ba4dbdb9694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.436023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae842290-bfd5-4740-b363-6ba4dbdb9694 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.436730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae842290-bfd5-4740-b363-6ba4dbdb9694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:29:48.437058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae842290-bfd5-4740-b363-6ba4dbdb9694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:48.438682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae842290-bfd5-4740-b363-6ba4dbdb9694 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:48.455172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1223/2449] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:29:48.456417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06927e4a-5697-451f-b996-9be7933b2009 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:48.466705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06927e4a-5697-451f-b996-9be7933b2009 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:48.472509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf355d2-5eae-47d4-98be-5ee29c949fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:48.473110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf355d2-5eae-47d4-98be-5ee29c949fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.473388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf355d2-5eae-47d4-98be-5ee29c949fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.473657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf355d2-5eae-47d4-98be-5ee29c949fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.474353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf355d2-5eae-47d4-98be-5ee29c949fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:48.476253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf355d2-5eae-47d4-98be-5ee29c949fbb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:48.487593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:48.506214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1227/2450] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:29:48 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:48.510846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae842290-bfd5-4740-b363-6ba4dbdb9694 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:48.521684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae842290-bfd5-4740-b363-6ba4dbdb9694 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:48.568211 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:48.582461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2548dc4-8e48-404e-9948-6cdc9b95dd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:48.583069 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2548dc4-8e48-404e-9948-6cdc9b95dd8d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.583349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2548dc4-8e48-404e-9948-6cdc9b95dd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.583622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2548dc4-8e48-404e-9948-6cdc9b95dd8d admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.584245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2548dc4-8e48-404e-9948-6cdc9b95dd8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:48.586204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2548dc4-8e48-404e-9948-6cdc9b95dd8d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:48.607539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1224/2451] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:29:48 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:29:48.612555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf355d2-5eae-47d4-98be-5ee29c949fbb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:48.623204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf355d2-5eae-47d4-98be-5ee29c949fbb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:48.628480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2301fe7-fafe-4591-b123-25a549b694bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:48.629019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2301fe7-fafe-4591-b123-25a549b694bd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.629295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2301fe7-fafe-4591-b123-25a549b694bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.629565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2301fe7-fafe-4591-b123-25a549b694bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.630204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2301fe7-fafe-4591-b123-25a549b694bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:48.631902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2301fe7-fafe-4591-b123-25a549b694bd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:48.642919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1228/2452] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:29:48.646826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2548dc4-8e48-404e-9948-6cdc9b95dd8d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:48.657848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2548dc4-8e48-404e-9948-6cdc9b95dd8d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:48.662437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325a61-8589-47b3-8a2c-4e7cfc4eaa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:48.663020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325a61-8589-47b3-8a2c-4e7cfc4eaa17 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.663297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325a61-8589-47b3-8a2c-4e7cfc4eaa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.663566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325a61-8589-47b3-8a2c-4e7cfc4eaa17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.664228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325a61-8589-47b3-8a2c-4e7cfc4eaa17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:48.665851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325a61-8589-47b3-8a2c-4e7cfc4eaa17 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:48.678890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1225/2453] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:29:48.682884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2301fe7-fafe-4591-b123-25a549b694bd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:48.693965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2301fe7-fafe-4591-b123-25a549b694bd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:48.698584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22c61d-bd8c-43f8-8fd4-d03b5dc136b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:48.699255 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22c61d-bd8c-43f8-8fd4-d03b5dc136b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.699533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22c61d-bd8c-43f8-8fd4-d03b5dc136b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.699807 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22c61d-bd8c-43f8-8fd4-d03b5dc136b6 admin admin] PATH_INFO: `/v3/projects/0e4d2afa381a4d00bc29027f244d8100/users/4b8138b6040041eab4f289a68e147fd9/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.714849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22c61d-bd8c-43f8-8fd4-d03b5dc136b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4d2afa381a4d00bc29027f244d8100, user_id=4b8138b6040041eab4f289a68e147fd9, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:48.717564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22c61d-bd8c-43f8-8fd4-d03b5dc136b6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:48.730110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1229/2454] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:29:48 2022] PUT /identity/v3/projects/0e4d2afa381a4d00bc29027f244d8100/users/4b8138b6040041eab4f289a68e147fd9/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:48.742777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6adca0-4c80-4677-b033-2288acb9764e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.743061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6adca0-4c80-4677-b033-2288acb9764e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.743336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6adca0-4c80-4677-b033-2288acb9764e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.772146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5c6adca0-4c80-4677-b033-2288acb9764e None None] MFA Rules not processed for user `4b8138b6040041eab4f289a68e147fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:48.773348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5c6adca0-4c80-4677-b033-2288acb9764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:48.836848 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1226/2455] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:48.841319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22c61d-bd8c-43f8-8fd4-d03b5dc136b6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:48.904144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22c61d-bd8c-43f8-8fd4-d03b5dc136b6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:48.909570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e233f-06df-4552-bc74-6cdcf20aa0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:48.910170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e233f-06df-4552-bc74-6cdcf20aa0ab admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:48.910451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e233f-06df-4552-bc74-6cdcf20aa0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:48.910724 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e233f-06df-4552-bc74-6cdcf20aa0ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:48.911355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e233f-06df-4552-bc74-6cdcf20aa0ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:48.913354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e233f-06df-4552-bc74-6cdcf20aa0ab admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:48.940256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1230/2456] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:29:48 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:48.944230 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6adca0-4c80-4677-b033-2288acb9764e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:48.954808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6adca0-4c80-4677-b033-2288acb9764e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:49.009665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bc554-af72-4835-806e-51a849a83f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:49.010257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bc554-af72-4835-806e-51a849a83f13 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:49.010535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bc554-af72-4835-806e-51a849a83f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:49.010805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bc554-af72-4835-806e-51a849a83f13 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:49.011415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bc554-af72-4835-806e-51a849a83f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:49.013385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bc554-af72-4835-806e-51a849a83f13 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:49.035710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1227/2457] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:29:48 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:49.039623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e233f-06df-4552-bc74-6cdcf20aa0ab admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:49.051061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e233f-06df-4552-bc74-6cdcf20aa0ab admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:49.055660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9dae-1407-4ec4-8020-ff9ef5209432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:49.056251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9dae-1407-4ec4-8020-ff9ef5209432 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:49.056600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9dae-1407-4ec4-8020-ff9ef5209432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:49.056977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9dae-1407-4ec4-8020-ff9ef5209432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:49.057617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f9dae-1407-4ec4-8020-ff9ef5209432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:49.059228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f9dae-1407-4ec4-8020-ff9ef5209432 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:49.069815 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1231/2458] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:29:49.073965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bc554-af72-4835-806e-51a849a83f13 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:49.084697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bc554-af72-4835-806e-51a849a83f13 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:49.089300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a8044-17ce-4cea-a85c-be6c1c45ed9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:49.089938 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a8044-17ce-4cea-a85c-be6c1c45ed9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:49.090210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a8044-17ce-4cea-a85c-be6c1c45ed9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:49.090482 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a8044-17ce-4cea-a85c-be6c1c45ed9c admin admin] PATH_INFO: `/v3/projects/c3c21779c663416d90bf2faa4e2077e5/users/ec36344bf00944e69f3db706f98d249b/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:49.105416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a8044-17ce-4cea-a85c-be6c1c45ed9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c21779c663416d90bf2faa4e2077e5, user_id=ec36344bf00944e69f3db706f98d249b, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:49.108094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a8044-17ce-4cea-a85c-be6c1c45ed9c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:49.121761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1228/2459] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:29:49 2022] PUT /identity/v3/projects/c3c21779c663416d90bf2faa4e2077e5/users/ec36344bf00944e69f3db706f98d249b/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:49.126198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9dae-1407-4ec4-8020-ff9ef5209432 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:49.136253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9dae-1407-4ec4-8020-ff9ef5209432 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:49.193057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73ee31-1a03-4664-a925-0cc5e2127cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:49.193662 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73ee31-1a03-4664-a925-0cc5e2127cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:49.193938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73ee31-1a03-4664-a925-0cc5e2127cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:49.194204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73ee31-1a03-4664-a925-0cc5e2127cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:49.194837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73ee31-1a03-4664-a925-0cc5e2127cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:49.196457 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73ee31-1a03-4664-a925-0cc5e2127cf3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:49.206744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1232/2460] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:29:49.211076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a8044-17ce-4cea-a85c-be6c1c45ed9c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:49.221940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a8044-17ce-4cea-a85c-be6c1c45ed9c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:49.226543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff5d82-4c58-454e-a949-ab43e4c986a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:49.227179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff5d82-4c58-454e-a949-ab43e4c986a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:49.227460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff5d82-4c58-454e-a949-ab43e4c986a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:49.227733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff5d82-4c58-454e-a949-ab43e4c986a1 admin admin] PATH_INFO: `/v3/projects/c3c21779c663416d90bf2faa4e2077e5/users/ec36344bf00944e69f3db706f98d249b/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:49.229634 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff5d82-4c58-454e-a949-ab43e4c986a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c21779c663416d90bf2faa4e2077e5, user_id=ec36344bf00944e69f3db706f98d249b, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:49.232288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff5d82-4c58-454e-a949-ab43e4c986a1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:49.242471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1229/2461] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:29:49 2022] PUT /identity/v3/projects/c3c21779c663416d90bf2faa4e2077e5/users/ec36344bf00944e69f3db706f98d249b/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:49.253428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304add1-017f-4c08-a591-85d59c89b2f7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:49.253908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304add1-017f-4c08-a591-85d59c89b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:49.254202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304add1-017f-4c08-a591-85d59c89b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:49.283099 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f304add1-017f-4c08-a591-85d59c89b2f7 None None] MFA Rules not processed for user `ec36344bf00944e69f3db706f98d249b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:49.284299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f304add1-017f-4c08-a591-85d59c89b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:49.296692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:49.362569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1233/2462] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:49.370138 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff5d82-4c58-454e-a949-ab43e4c986a1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:49.371878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bbff5d82-4c58-454e-a949-ab43e4c986a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:49.382645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:49.433084 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff5d82-4c58-454e-a949-ab43e4c986a1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:49.436903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d5698-9bed-405a-9141-13b0841ab6dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:49.438006 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d5698-9bed-405a-9141-13b0841ab6dd service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:49.438295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d5698-9bed-405a-9141-13b0841ab6dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:49.438560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d5698-9bed-405a-9141-13b0841ab6dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:49.447565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d5698-9bed-405a-9141-13b0841ab6dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:49.449029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d5698-9bed-405a-9141-13b0841ab6dd service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:49.458937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1230/2463] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:49.877740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304add1-017f-4c08-a591-85d59c89b2f7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:49.941788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304add1-017f-4c08-a591-85d59c89b2f7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:49.946322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:49.946946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:49.947223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:49.947493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] PATH_INFO: `/v3/users/4b8138b6040041eab4f289a68e147fd9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:49.948630 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b8138b6040041eab4f289a68e147fd9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:49.950545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:49.998042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b8138b6040041eab4f289a68e147fd9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:49.998694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b8138b6040041eab4f289a68e147fd9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:49.999299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b8138b6040041eab4f289a68e147fd9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.006793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b8138b6040041eab4f289a68e147fd9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.019324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b8138b6040041eab4f289a68e147fd9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.032399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1234/2464] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:29:49 2022] DELETE /identity/v3/users/4b8138b6040041eab4f289a68e147fd9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:50.036979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da96c0-178a-4495-a923-634182d43201 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:50.037502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da96c0-178a-4495-a923-634182d43201 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:50.037733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da96c0-178a-4495-a923-634182d43201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:50.057257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-58da96c0-178a-4495-a923-634182d43201 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:50.058729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-58da96c0-178a-4495-a923-634182d43201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:50.127664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1231/2465] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:50.135211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:50.136644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:50.176314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:50.213849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6ebb2-e886-41a1-a2e0-01f24dc95afc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:50.217571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4311ec4-3ccf-4371-accf-7f2a50325ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:50.218184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4311ec4-3ccf-4371-accf-7f2a50325ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:50.218463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4311ec4-3ccf-4371-accf-7f2a50325ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:50.218831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4311ec4-3ccf-4371-accf-7f2a50325ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:50.228020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4311ec4-3ccf-4371-accf-7f2a50325ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:50.229537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4311ec4-3ccf-4371-accf-7f2a50325ef0 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:50.240717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1235/2466] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:50.520142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da96c0-178a-4495-a923-634182d43201 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:50.521711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-58da96c0-178a-4495-a923-634182d43201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:50.536465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da96c0-178a-4495-a923-634182d43201 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:50.540988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:50.541614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:50.541887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:50.542666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] PATH_INFO: `/v3/projects/0e4d2afa381a4d00bc29027f244d8100` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:50.543949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e4d2afa381a4d00bc29027f244d8100)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:50.545894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:50.594879 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e4d2afa381a4d00bc29027f244d8100', 'request_id': 'req-580d3c8d-18de-430b-8523-b1437053fcf0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.600220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e4d2afa381a4d00bc29027f244d8100', 'request_id': 'req-580d3c8d-18de-430b-8523-b1437053fcf0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.607343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e4d2afa381a4d00bc29027f244d8100', 'request_id': 'req-580d3c8d-18de-430b-8523-b1437053fcf0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.619590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] The token cache is being invalidate because project 0e4d2afa381a4d00bc29027f244d8100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:29:50.619885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.620563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.622072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1232/2467] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:29:50 2022] DELETE /identity/v3/projects/0e4d2afa381a4d00bc29027f244d8100 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:50.626764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4311ec4-3ccf-4371-accf-7f2a50325ef0 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:50.638362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4311ec4-3ccf-4371-accf-7f2a50325ef0 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:50.698287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:50.698937 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:50.699199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:50.699488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] PATH_INFO: `/v3/users/ec36344bf00944e69f3db706f98d249b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:50.700621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec36344bf00944e69f3db706f98d249b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:50.702582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:50.747692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec36344bf00944e69f3db706f98d249b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.748354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec36344bf00944e69f3db706f98d249b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.748979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec36344bf00944e69f3db706f98d249b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.756822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec36344bf00944e69f3db706f98d249b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.768824 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c28aad4a-b075-4027-81c5-26b7c810dde4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec36344bf00944e69f3db706f98d249b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:50.781580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1236/2468] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:29:50 2022] DELETE /identity/v3/users/ec36344bf00944e69f3db706f98d249b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:29:51.061994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:51.064314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:51.080557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d3c8d-18de-430b-8523-b1437053fcf0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:51.112658 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:51.143148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:51.143705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.143986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.144261 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] PATH_INFO: `/v3/projects/c3c21779c663416d90bf2faa4e2077e5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.145414 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c21779c663416d90bf2faa4e2077e5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:51.147347 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:51.193623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c21779c663416d90bf2faa4e2077e5', 'request_id': 'req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:51.199045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c21779c663416d90bf2faa4e2077e5', 'request_id': 'req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:51.206323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c21779c663416d90bf2faa4e2077e5', 'request_id': 'req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:51.217439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] The token cache is being invalidate because project c3c21779c663416d90bf2faa4e2077e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:29:51.218086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:51.218345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-db4f4c02-6ce5-46dd-9475-18ab8b8918e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:51.220532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1233/2469] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:29:51 2022] DELETE /identity/v3/projects/c3c21779c663416d90bf2faa4e2077e5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:51.225415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8267d-ee20-4f18-a6ef-48e1c40449f0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.225870 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8267d-ee20-4f18-a6ef-48e1c40449f0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.225980 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8267d-ee20-4f18-a6ef-48e1c40449f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.246145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aea8267d-ee20-4f18-a6ef-48e1c40449f0 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:51.247352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aea8267d-ee20-4f18-a6ef-48e1c40449f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:51.302344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:51.318363 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1237/2470] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:51.326264 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6f106-7af3-40de-964b-3445fba756c3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.326557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6f106-7af3-40de-964b-3445fba756c3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.326836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6f106-7af3-40de-964b-3445fba756c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.347546 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d4a6f106-7af3-40de-964b-3445fba756c3 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:51.348785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d4a6f106-7af3-40de-964b-3445fba756c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:51.356989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1234/2471] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:51.361226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8267d-ee20-4f18-a6ef-48e1c40449f0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:51.371056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8267d-ee20-4f18-a6ef-48e1c40449f0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:51.375968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2fa4a-5b73-49e0-a9bf-64af4b938972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:51.376610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2fa4a-5b73-49e0-a9bf-64af4b938972 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.376896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2fa4a-5b73-49e0-a9bf-64af4b938972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.377173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2fa4a-5b73-49e0-a9bf-64af4b938972 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.377831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2fa4a-5b73-49e0-a9bf-64af4b938972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:29:51.378136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2fa4a-5b73-49e0-a9bf-64af4b938972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:51.379764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2fa4a-5b73-49e0-a9bf-64af4b938972 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:51.392226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1238/2472] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:29:51.396669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6f106-7af3-40de-964b-3445fba756c3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:51.405878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6f106-7af3-40de-964b-3445fba756c3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:51.411302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502457b4-4217-4759-af93-916934d37428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:51.411946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502457b4-4217-4759-af93-916934d37428 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.412315 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502457b4-4217-4759-af93-916934d37428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.412531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502457b4-4217-4759-af93-916934d37428 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.413176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502457b4-4217-4759-af93-916934d37428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:51.415154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502457b4-4217-4759-af93-916934d37428 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:51.448495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1235/2473] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:29:51 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:51.449069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2fa4a-5b73-49e0-a9bf-64af4b938972 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:51.460217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2fa4a-5b73-49e0-a9bf-64af4b938972 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:51.523860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ed080-9c5a-4a15-83e0-f4e5b9c7ab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:51.524184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ed080-9c5a-4a15-83e0-f4e5b9c7ab3e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.524493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ed080-9c5a-4a15-83e0-f4e5b9c7ab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.524763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ed080-9c5a-4a15-83e0-f4e5b9c7ab3e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.525394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ed080-9c5a-4a15-83e0-f4e5b9c7ab3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:51.527358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ed080-9c5a-4a15-83e0-f4e5b9c7ab3e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:51.550512 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1239/2474] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:29:51 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:51.554482 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502457b4-4217-4759-af93-916934d37428 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:51.569792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502457b4-4217-4759-af93-916934d37428 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:51.575437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3c663-7327-4bde-8f8a-d6c8f4672a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:51.576508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3c663-7327-4bde-8f8a-d6c8f4672a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.576808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3c663-7327-4bde-8f8a-d6c8f4672a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.577063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3c663-7327-4bde-8f8a-d6c8f4672a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.577729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3c663-7327-4bde-8f8a-d6c8f4672a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:51.579856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3c663-7327-4bde-8f8a-d6c8f4672a40 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:51.594803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1236/2475] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:29:51.598892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ed080-9c5a-4a15-83e0-f4e5b9c7ab3e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:51.609266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ed080-9c5a-4a15-83e0-f4e5b9c7ab3e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:51.617077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bdcc02-660f-4927-a2c2-3bf4fd195c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:51.617683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdcc02-660f-4927-a2c2-3bf4fd195c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.617958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdcc02-660f-4927-a2c2-3bf4fd195c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.618222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bdcc02-660f-4927-a2c2-3bf4fd195c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.630690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdcc02-660f-4927-a2c2-3bf4fd195c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:51.630690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bdcc02-660f-4927-a2c2-3bf4fd195c52 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:51.633916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1240/2476] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:29:51.638439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3c663-7327-4bde-8f8a-d6c8f4672a40 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:51.648331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3c663-7327-4bde-8f8a-d6c8f4672a40 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:51.653313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16ce4-215d-410f-a636-e5ef9d392d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:51.653962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16ce4-215d-410f-a636-e5ef9d392d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.654243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16ce4-215d-410f-a636-e5ef9d392d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.654655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c16ce4-215d-410f-a636-e5ef9d392d2d admin admin] PATH_INFO: `/v3/projects/d87f85863c204ad4943ca670ee970c11/users/e10878be2767432d89f5c289c0cc1135/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.671428 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c16ce4-215d-410f-a636-e5ef9d392d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87f85863c204ad4943ca670ee970c11, user_id=e10878be2767432d89f5c289c0cc1135, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:51.675907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c16ce4-215d-410f-a636-e5ef9d392d2d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:51.690789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1237/2477] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:29:51 2022] PUT /identity/v3/projects/d87f85863c204ad4943ca670ee970c11/users/e10878be2767432d89f5c289c0cc1135/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:51.702241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15528-ff1a-4524-bb20-3727766d13e1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.702241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15528-ff1a-4524-bb20-3727766d13e1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.702450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f15528-ff1a-4524-bb20-3727766d13e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.731340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-20f15528-ff1a-4524-bb20-3727766d13e1 None None] MFA Rules not processed for user `e10878be2767432d89f5c289c0cc1135`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:51.732589 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-20f15528-ff1a-4524-bb20-3727766d13e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:51.798914 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1241/2478] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:29:51.803843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16ce4-215d-410f-a636-e5ef9d392d2d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:51.866093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c16ce4-215d-410f-a636-e5ef9d392d2d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:51.870923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adf569-a916-4855-a21b-9090038a97f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:51.871526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adf569-a916-4855-a21b-9090038a97f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.871806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adf569-a916-4855-a21b-9090038a97f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.872167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adf569-a916-4855-a21b-9090038a97f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.872806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adf569-a916-4855-a21b-9090038a97f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:51.875154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adf569-a916-4855-a21b-9090038a97f7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:51.880186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:51.904594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1238/2479] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:29:51 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:51.909177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15528-ff1a-4524-bb20-3727766d13e1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:51.919657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f15528-ff1a-4524-bb20-3727766d13e1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:51.966624 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:51.980329 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5c4c1-3c5b-4722-a342-0e025aaf27d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:51.981015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5c4c1-3c5b-4722-a342-0e025aaf27d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:51.981296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5c4c1-3c5b-4722-a342-0e025aaf27d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:51.981567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5c4c1-3c5b-4722-a342-0e025aaf27d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:51.982206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5c4c1-3c5b-4722-a342-0e025aaf27d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:51.984225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5c4c1-3c5b-4722-a342-0e025aaf27d1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:52.006231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1242/2480] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:29:51 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:52.010304 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adf569-a916-4855-a21b-9090038a97f7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:52.021404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adf569-a916-4855-a21b-9090038a97f7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:52.026022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601e663-adf4-49c7-81c6-7c2d12a80459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:52.026614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601e663-adf4-49c7-81c6-7c2d12a80459 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.026894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601e663-adf4-49c7-81c6-7c2d12a80459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.027166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601e663-adf4-49c7-81c6-7c2d12a80459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.027805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601e663-adf4-49c7-81c6-7c2d12a80459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:52.029447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601e663-adf4-49c7-81c6-7c2d12a80459 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:52.040767 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1239/2481] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:29:52.045416 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5c4c1-3c5b-4722-a342-0e025aaf27d1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:52.057772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5c4c1-3c5b-4722-a342-0e025aaf27d1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:52.062345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f76265-9f0d-4cde-bdbe-57acff48fd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:52.063575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f76265-9f0d-4cde-bdbe-57acff48fd94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.063853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f76265-9f0d-4cde-bdbe-57acff48fd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.064122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f76265-9f0d-4cde-bdbe-57acff48fd94 admin admin] PATH_INFO: `/v3/projects/6dbf2d0019424813a4b7efa849c7eed0/users/513ab23334f74818a0f5da25a9f6c415/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.079757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f76265-9f0d-4cde-bdbe-57acff48fd94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dbf2d0019424813a4b7efa849c7eed0, user_id=513ab23334f74818a0f5da25a9f6c415, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:52.082486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f76265-9f0d-4cde-bdbe-57acff48fd94 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:52.095299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1243/2482] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:29:52 2022] PUT /identity/v3/projects/6dbf2d0019424813a4b7efa849c7eed0/users/513ab23334f74818a0f5da25a9f6c415/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:29:52.099422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601e663-adf4-49c7-81c6-7c2d12a80459 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:52.109287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601e663-adf4-49c7-81c6-7c2d12a80459 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:52.166965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1bec9-1797-4109-b4d0-f436c86b5b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:52.167490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1bec9-1797-4109-b4d0-f436c86b5b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.167775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1bec9-1797-4109-b4d0-f436c86b5b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.168049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1bec9-1797-4109-b4d0-f436c86b5b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.168718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1bec9-1797-4109-b4d0-f436c86b5b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:52.170345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1bec9-1797-4109-b4d0-f436c86b5b50 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:52.180106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1240/2483] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:29:52.184521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f76265-9f0d-4cde-bdbe-57acff48fd94 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:52.194977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f76265-9f0d-4cde-bdbe-57acff48fd94 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:52.199856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212b82b-ee58-494e-85d1-97ec402aad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:52.200681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212b82b-ee58-494e-85d1-97ec402aad8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.200960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212b82b-ee58-494e-85d1-97ec402aad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.201235 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212b82b-ee58-494e-85d1-97ec402aad8f admin admin] PATH_INFO: `/v3/projects/6dbf2d0019424813a4b7efa849c7eed0/users/513ab23334f74818a0f5da25a9f6c415/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.203677 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212b82b-ee58-494e-85d1-97ec402aad8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dbf2d0019424813a4b7efa849c7eed0, user_id=513ab23334f74818a0f5da25a9f6c415, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:52.206487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212b82b-ee58-494e-85d1-97ec402aad8f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:52.217019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1244/2484] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:29:52 2022] PUT /identity/v3/projects/6dbf2d0019424813a4b7efa849c7eed0/users/513ab23334f74818a0f5da25a9f6c415/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:52.228930 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375424ce-4ade-4cad-9396-7049404f87c2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.229222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375424ce-4ade-4cad-9396-7049404f87c2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.229501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375424ce-4ade-4cad-9396-7049404f87c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.257717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-375424ce-4ade-4cad-9396-7049404f87c2 None None] MFA Rules not processed for user `513ab23334f74818a0f5da25a9f6c415`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:52.258974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-375424ce-4ade-4cad-9396-7049404f87c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:52.327194 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1241/2485] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:52.339971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212b82b-ee58-494e-85d1-97ec402aad8f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:52.341806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9212b82b-ee58-494e-85d1-97ec402aad8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:52.399734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212b82b-ee58-494e-85d1-97ec402aad8f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:52.403391 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132bbf8-08f1-43dc-bbee-d49e7e7b0da5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:52.403989 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132bbf8-08f1-43dc-bbee-d49e7e7b0da5 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.404277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132bbf8-08f1-43dc-bbee-d49e7e7b0da5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.404597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132bbf8-08f1-43dc-bbee-d49e7e7b0da5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.414222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132bbf8-08f1-43dc-bbee-d49e7e7b0da5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:52.415678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132bbf8-08f1-43dc-bbee-d49e7e7b0da5 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:52.425367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1245/2486] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:52.748308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375424ce-4ade-4cad-9396-7049404f87c2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:52.757673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375424ce-4ade-4cad-9396-7049404f87c2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:52.762125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b1eb7-8500-4fe7-8f10-0968a53d2df0 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '513ab23334f74818a0f5da25a9f6c415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dbf2d0019424813a4b7efa849c7eed0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:52.762711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b1eb7-8500-4fe7-8f10-0968a53d2df0 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.762988 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b1eb7-8500-4fe7-8f10-0968a53d2df0 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.763256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b1eb7-8500-4fe7-8f10-0968a53d2df0 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.763897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b1eb7-8500-4fe7-8f10-0968a53d2df0 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:29:52.764244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b1eb7-8500-4fe7-8f10-0968a53d2df0 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:52.765955 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b1eb7-8500-4fe7-8f10-0968a53d2df0 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:52.781440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1242/2487] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:29:52.786792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132bbf8-08f1-43dc-bbee-d49e7e7b0da5 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:52.796006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132bbf8-08f1-43dc-bbee-d49e7e7b0da5 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:52.800969 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a17143-c422-4376-8baa-cf157b0da21c tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '513ab23334f74818a0f5da25a9f6c415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dbf2d0019424813a4b7efa849c7eed0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:52.801560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a17143-c422-4376-8baa-cf157b0da21c tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.801851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a17143-c422-4376-8baa-cf157b0da21c tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.802136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a17143-c422-4376-8baa-cf157b0da21c tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.802765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a17143-c422-4376-8baa-cf157b0da21c tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:52.804726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a17143-c422-4376-8baa-cf157b0da21c tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:52.832917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1246/2488] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:29:52 2022] POST /identity/v3/projects => generated 353 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:29:52.837034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b1eb7-8500-4fe7-8f10-0968a53d2df0 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:52.847602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b1eb7-8500-4fe7-8f10-0968a53d2df0 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:52.886939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:52.904600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '513ab23334f74818a0f5da25a9f6c415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dbf2d0019424813a4b7efa849c7eed0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:52.905208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.905492 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.905772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.906421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:29:52.906730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:52.908939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:52.920756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1243/2489] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:29:52.958560 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a17143-c422-4376-8baa-cf157b0da21c tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:52.971859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a17143-c422-4376-8baa-cf157b0da21c tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:52.977562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '513ab23334f74818a0f5da25a9f6c415', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6dbf2d0019424813a4b7efa849c7eed0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:52.978188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:52.978471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:52.978744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] PATH_INFO: `/v3/projects/ffef6829f5c744588b66132a1d26d084` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:52.979859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ffef6829f5c744588b66132a1d26d084)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:52.981934 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:53.027274 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffef6829f5c744588b66132a1d26d084', 'request_id': 'req-e67b4c72-0361-4b32-b351-46ad77288e68'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.033437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffef6829f5c744588b66132a1d26d084', 'request_id': 'req-e67b4c72-0361-4b32-b351-46ad77288e68'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.040938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffef6829f5c744588b66132a1d26d084', 'request_id': 'req-e67b4c72-0361-4b32-b351-46ad77288e68'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.053110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] The token cache is being invalidate because project ffef6829f5c744588b66132a1d26d084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:29:53.053404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.054054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e67b4c72-0361-4b32-b351-46ad77288e68 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.055517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1247/2490] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:29:52 2022] DELETE /identity/v3/projects/ffef6829f5c744588b66132a1d26d084 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:53.247631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:53.249361 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:53.313191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9955d-d370-4728-8094-756a3da7d1b1 tempest-VolumeQuotaClassesTest-1206454492 tempest-VolumeQuotaClassesTest-1206454492-project-admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:53.371302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:53.371928 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:53.372210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:53.372551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] PATH_INFO: `/v3/users/e10878be2767432d89f5c289c0cc1135` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:53.373669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e10878be2767432d89f5c289c0cc1135)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:53.375610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:53.422533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e10878be2767432d89f5c289c0cc1135'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.430274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e10878be2767432d89f5c289c0cc1135'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.430991 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e10878be2767432d89f5c289c0cc1135'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.443684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e10878be2767432d89f5c289c0cc1135'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.450724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e10878be2767432d89f5c289c0cc1135'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:53.457353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1244/2491] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:29:53 2022] DELETE /identity/v3/users/e10878be2767432d89f5c289c0cc1135 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:53.461805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd26d7-037e-4600-b83b-72751abe7c74 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:53.462294 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd26d7-037e-4600-b83b-72751abe7c74 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:53.462414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd26d7-037e-4600-b83b-72751abe7c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:53.470406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:53.482867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-89fd26d7-037e-4600-b83b-72751abe7c74 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:53.484017 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-89fd26d7-037e-4600-b83b-72751abe7c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:53.541627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1248/2492] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:29:53.552156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:53.553603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:53.558884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:53.614614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b781a64-8faf-4203-869f-3cba8a25670f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:53.618376 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8efce4-0419-4ec2-9929-e467b91698c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:53.618975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8efce4-0419-4ec2-9929-e467b91698c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:53.619254 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8efce4-0419-4ec2-9929-e467b91698c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:53.619537 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8efce4-0419-4ec2-9929-e467b91698c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:53.629455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8efce4-0419-4ec2-9929-e467b91698c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:53.630894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8efce4-0419-4ec2-9929-e467b91698c1 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:53.640152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1245/2493] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:53.926482 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd26d7-037e-4600-b83b-72751abe7c74 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:53.927921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-89fd26d7-037e-4600-b83b-72751abe7c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:53.941161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd26d7-037e-4600-b83b-72751abe7c74 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:53.945788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:53.946425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:53.946707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:53.946991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] PATH_INFO: `/v3/projects/d87f85863c204ad4943ca670ee970c11` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:53.948134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d87f85863c204ad4943ca670ee970c11)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:53.950134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:54.007755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd87f85863c204ad4943ca670ee970c11', 'request_id': 'req-a2828928-d44d-41c6-a256-7c0c36e4ccf8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.014227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd87f85863c204ad4943ca670ee970c11', 'request_id': 'req-a2828928-d44d-41c6-a256-7c0c36e4ccf8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.023303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd87f85863c204ad4943ca670ee970c11', 'request_id': 'req-a2828928-d44d-41c6-a256-7c0c36e4ccf8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.035507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] The token cache is being invalidate because project d87f85863c204ad4943ca670ee970c11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:29:54.035798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.036559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.038090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1249/2494] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:29:53 2022] DELETE /identity/v3/projects/d87f85863c204ad4943ca670ee970c11 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:54.042447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8efce4-0419-4ec2-9929-e467b91698c1 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:54.053302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8efce4-0419-4ec2-9929-e467b91698c1 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:54.114165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:54.114791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:54.115076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:54.115345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] PATH_INFO: `/v3/users/513ab23334f74818a0f5da25a9f6c415` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:54.116654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=513ab23334f74818a0f5da25a9f6c415)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:54.118985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:54.164827 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513ab23334f74818a0f5da25a9f6c415'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.171891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513ab23334f74818a0f5da25a9f6c415'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.172585 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513ab23334f74818a0f5da25a9f6c415'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.184559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513ab23334f74818a0f5da25a9f6c415'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.192325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d562f450-1ff9-4826-aae1-6d99ce511665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513ab23334f74818a0f5da25a9f6c415'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.198120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1246/2495] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:29:54 2022] DELETE /identity/v3/users/513ab23334f74818a0f5da25a9f6c415 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:29:54.468523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:54.470792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:54.484698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2828928-d44d-41c6-a256-7c0c36e4ccf8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:54.542360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:54.543034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:54.543257 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:54.543523 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] PATH_INFO: `/v3/projects/6dbf2d0019424813a4b7efa849c7eed0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:54.544659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dbf2d0019424813a4b7efa849c7eed0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:54.546688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:54.580822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:54.594511 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dbf2d0019424813a4b7efa849c7eed0', 'request_id': 'req-be04ff36-5cf0-461e-a412-22aa825f4c57'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.600187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dbf2d0019424813a4b7efa849c7eed0', 'request_id': 'req-be04ff36-5cf0-461e-a412-22aa825f4c57'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.606671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dbf2d0019424813a4b7efa849c7eed0', 'request_id': 'req-be04ff36-5cf0-461e-a412-22aa825f4c57'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.618449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] The token cache is being invalidate because project 6dbf2d0019424813a4b7efa849c7eed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:29:54.618732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.619355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be04ff36-5cf0-461e-a412-22aa825f4c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:54.620855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1250/2496] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:29:54 2022] DELETE /identity/v3/projects/6dbf2d0019424813a4b7efa849c7eed0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:54.626996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f3ed4-f800-4293-b8f8-45e7c77eda78 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:54.627518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f3ed4-f800-4293-b8f8-45e7c77eda78 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:54.627616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f3ed4-f800-4293-b8f8-45e7c77eda78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:54.646660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9c4f3ed4-f800-4293-b8f8-45e7c77eda78 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:54.647855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9c4f3ed4-f800-4293-b8f8-45e7c77eda78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:54.666583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:54.711565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1247/2497] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:54.721297 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692870-a99f-4dca-a035-ccf9fd3dfe15 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:54.721632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692870-a99f-4dca-a035-ccf9fd3dfe15 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:54.721904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b692870-a99f-4dca-a035-ccf9fd3dfe15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:54.742155 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2b692870-a99f-4dca-a035-ccf9fd3dfe15 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:54.743342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2b692870-a99f-4dca-a035-ccf9fd3dfe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:54.751225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1251/2498] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:54.755741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f3ed4-f800-4293-b8f8-45e7c77eda78 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:54.765654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f3ed4-f800-4293-b8f8-45e7c77eda78 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:54.770590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9702c4f-adae-4cf5-b334-fd2161bca4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:54.771225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9702c4f-adae-4cf5-b334-fd2161bca4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:54.771513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9702c4f-adae-4cf5-b334-fd2161bca4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:54.771784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9702c4f-adae-4cf5-b334-fd2161bca4e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:54.772519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9702c4f-adae-4cf5-b334-fd2161bca4e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:29:54.772825 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9702c4f-adae-4cf5-b334-fd2161bca4e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:54.774467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9702c4f-adae-4cf5-b334-fd2161bca4e2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:54.787003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1248/2499] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:29:54.791748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692870-a99f-4dca-a035-ccf9fd3dfe15 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:54.802879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b692870-a99f-4dca-a035-ccf9fd3dfe15 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:54.807540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d07cc-d604-4670-a3b4-1aeb6b16f82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:54.808134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d07cc-d604-4670-a3b4-1aeb6b16f82b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:54.808434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d07cc-d604-4670-a3b4-1aeb6b16f82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:54.808709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d07cc-d604-4670-a3b4-1aeb6b16f82b admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:54.809323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d07cc-d604-4670-a3b4-1aeb6b16f82b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:54.811217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d07cc-d604-4670-a3b4-1aeb6b16f82b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:54.840650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1252/2500] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:29:54 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:54.841785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9702c4f-adae-4cf5-b334-fd2161bca4e2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:54.853073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9702c4f-adae-4cf5-b334-fd2161bca4e2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:54.914097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b002d-5c86-48bd-98a8-89d8c3395c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:54.914698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b002d-5c86-48bd-98a8-89d8c3395c35 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:54.914978 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b002d-5c86-48bd-98a8-89d8c3395c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:54.915249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b002d-5c86-48bd-98a8-89d8c3395c35 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:54.915878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b002d-5c86-48bd-98a8-89d8c3395c35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:54.917847 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b002d-5c86-48bd-98a8-89d8c3395c35 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:54.939120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1249/2501] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:29:54 2022] POST /identity/v3/users => generated 281 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:54.943316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d07cc-d604-4670-a3b4-1aeb6b16f82b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:54.953561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d07cc-d604-4670-a3b4-1aeb6b16f82b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:54.958440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcce2f-4d0c-4289-b26f-b5667a360891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:54.959025 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcce2f-4d0c-4289-b26f-b5667a360891 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:54.959299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcce2f-4d0c-4289-b26f-b5667a360891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:54.959564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcce2f-4d0c-4289-b26f-b5667a360891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:54.960192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcce2f-4d0c-4289-b26f-b5667a360891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:54.961829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcce2f-4d0c-4289-b26f-b5667a360891 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:54.972169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1253/2502] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:29:54.975967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b002d-5c86-48bd-98a8-89d8c3395c35 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:54.985768 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b002d-5c86-48bd-98a8-89d8c3395c35 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:54.991179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed47d2-fa18-470a-b619-c44cd05056b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:54.991757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed47d2-fa18-470a-b619-c44cd05056b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:54.992032 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed47d2-fa18-470a-b619-c44cd05056b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:54.992301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed47d2-fa18-470a-b619-c44cd05056b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:54.992967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed47d2-fa18-470a-b619-c44cd05056b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:54.994553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed47d2-fa18-470a-b619-c44cd05056b2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:55.004623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1250/2503] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:29:55.008642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcce2f-4d0c-4289-b26f-b5667a360891 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:55.019215 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcce2f-4d0c-4289-b26f-b5667a360891 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:55.023828 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f642578-078f-49ea-8b5b-e3def4eb65ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:55.024505 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f642578-078f-49ea-8b5b-e3def4eb65ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:55.024784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f642578-078f-49ea-8b5b-e3def4eb65ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:55.025057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f642578-078f-49ea-8b5b-e3def4eb65ae admin admin] PATH_INFO: `/v3/projects/aaf293e4cf85415cb41aaa292a7b0c56/users/cff6d71ba66c4e15af0d6996f667f234/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:55.040074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f642578-078f-49ea-8b5b-e3def4eb65ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf293e4cf85415cb41aaa292a7b0c56, user_id=cff6d71ba66c4e15af0d6996f667f234, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:55.042794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f642578-078f-49ea-8b5b-e3def4eb65ae admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:55.056322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1254/2504] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:29:55 2022] PUT /identity/v3/projects/aaf293e4cf85415cb41aaa292a7b0c56/users/cff6d71ba66c4e15af0d6996f667f234/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:55.066544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e274f3a-7d62-42de-b5a5-3a24b952aca0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:55.066832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e274f3a-7d62-42de-b5a5-3a24b952aca0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:55.067107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e274f3a-7d62-42de-b5a5-3a24b952aca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:55.095012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4e274f3a-7d62-42de-b5a5-3a24b952aca0 None None] MFA Rules not processed for user `cff6d71ba66c4e15af0d6996f667f234`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:55.096255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4e274f3a-7d62-42de-b5a5-3a24b952aca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:55.162233 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1251/2505] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:55.196420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f642578-078f-49ea-8b5b-e3def4eb65ae admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:55.198119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6f642578-078f-49ea-8b5b-e3def4eb65ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:55.255073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f642578-078f-49ea-8b5b-e3def4eb65ae admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:55.258700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17a17f-8e59-4e9b-96a7-f775b58c0100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:55.259299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a17f-8e59-4e9b-96a7-f775b58c0100 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:55.259580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a17f-8e59-4e9b-96a7-f775b58c0100 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:55.259852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a17f-8e59-4e9b-96a7-f775b58c0100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:55.268508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17a17f-8e59-4e9b-96a7-f775b58c0100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:55.270015 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17a17f-8e59-4e9b-96a7-f775b58c0100 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:55.279283 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1255/2506] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:55.295065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e274f3a-7d62-42de-b5a5-3a24b952aca0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:55.326399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:55.369568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e274f3a-7d62-42de-b5a5-3a24b952aca0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:55.374206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:55.374823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:55.375103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:55.375375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] PATH_INFO: `/v3/users/cff6d71ba66c4e15af0d6996f667f234` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:55.377257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff6d71ba66c4e15af0d6996f667f234)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:55.379184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:55.424259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff6d71ba66c4e15af0d6996f667f234'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:55.431465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff6d71ba66c4e15af0d6996f667f234'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:55.432124 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff6d71ba66c4e15af0d6996f667f234'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:55.443846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff6d71ba66c4e15af0d6996f667f234'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:55.451101 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff6d71ba66c4e15af0d6996f667f234'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:55.457148 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1252/2507] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:29:55 2022] DELETE /identity/v3/users/cff6d71ba66c4e15af0d6996f667f234 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:55.461809 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8efca-8dff-4b91-a5ed-f7b7810c69da None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:55.462099 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8efca-8dff-4b91-a5ed-f7b7810c69da None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:55.462380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8efca-8dff-4b91-a5ed-f7b7810c69da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:55.482144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-53b8efca-8dff-4b91-a5ed-f7b7810c69da None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:55.483365 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-53b8efca-8dff-4b91-a5ed-f7b7810c69da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:55.533362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:55.544166 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1256/2508] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:55.554337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:55.555840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:55.612724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e28f36-8e66-4ec1-ad04-1c57964e4eeb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:55.616209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12286fd-e650-4275-912d-d391d4111504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:55.616835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12286fd-e650-4275-912d-d391d4111504 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:55.617114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12286fd-e650-4275-912d-d391d4111504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:55.617386 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12286fd-e650-4275-912d-d391d4111504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:55.626244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12286fd-e650-4275-912d-d391d4111504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:55.628310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12286fd-e650-4275-912d-d391d4111504 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:55.637771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1253/2509] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:29:55.918236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8efca-8dff-4b91-a5ed-f7b7810c69da None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:55.919665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-53b8efca-8dff-4b91-a5ed-f7b7810c69da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:55.932857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8efca-8dff-4b91-a5ed-f7b7810c69da None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:55.937524 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:55.938151 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:55.938426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:55.938719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] PATH_INFO: `/v3/projects/aaf293e4cf85415cb41aaa292a7b0c56` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:55.940452 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf293e4cf85415cb41aaa292a7b0c56)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:55.943090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:55.999292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf293e4cf85415cb41aaa292a7b0c56', 'request_id': 'req-267a5c08-4f17-4ade-bf6d-c7d8def4b031'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:56.005054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf293e4cf85415cb41aaa292a7b0c56', 'request_id': 'req-267a5c08-4f17-4ade-bf6d-c7d8def4b031'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:56.012154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf293e4cf85415cb41aaa292a7b0c56', 'request_id': 'req-267a5c08-4f17-4ade-bf6d-c7d8def4b031'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:56.023805 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] The token cache is being invalidate because project aaf293e4cf85415cb41aaa292a7b0c56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:29:56.024094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:56.024765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-267a5c08-4f17-4ade-bf6d-c7d8def4b031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:29:56.026237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1257/2510] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:29:55 2022] DELETE /identity/v3/projects/aaf293e4cf85415cb41aaa292a7b0c56 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:56.032058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769ebb8-46eb-4076-b65f-7b4d36871107 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.032334 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769ebb8-46eb-4076-b65f-7b4d36871107 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.032636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769ebb8-46eb-4076-b65f-7b4d36871107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.052512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2769ebb8-46eb-4076-b65f-7b4d36871107 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:56.053690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2769ebb8-46eb-4076-b65f-7b4d36871107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:56.090715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:29:56.113514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1254/2511] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:56.121354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732e233-5675-49f9-847f-c9f670af4dbf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.121668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732e233-5675-49f9-847f-c9f670af4dbf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.121946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732e233-5675-49f9-847f-c9f670af4dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.142324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3732e233-5675-49f9-847f-c9f670af4dbf None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:56.144501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3732e233-5675-49f9-847f-c9f670af4dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:56.152282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1258/2512] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:56.156528 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769ebb8-46eb-4076-b65f-7b4d36871107 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:56.166357 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769ebb8-46eb-4076-b65f-7b4d36871107 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:56.170859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a7192-9360-468a-8556-f2732181de37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:56.171444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7192-9360-468a-8556-f2732181de37 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.171722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7192-9360-468a-8556-f2732181de37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.171993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7192-9360-468a-8556-f2732181de37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.172669 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a7192-9360-468a-8556-f2732181de37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:29:56.172970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a7192-9360-468a-8556-f2732181de37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:56.174724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a7192-9360-468a-8556-f2732181de37 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:56.186351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1255/2513] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:29:56.191406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732e233-5675-49f9-847f-c9f670af4dbf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:56.200461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732e233-5675-49f9-847f-c9f670af4dbf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:56.204971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5db53-bbcf-4b44-b9eb-1a4e1adf3db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:56.206132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5db53-bbcf-4b44-b9eb-1a4e1adf3db1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.206577 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5db53-bbcf-4b44-b9eb-1a4e1adf3db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.206849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5db53-bbcf-4b44-b9eb-1a4e1adf3db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.207462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5db53-bbcf-4b44-b9eb-1a4e1adf3db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:56.209393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5db53-bbcf-4b44-b9eb-1a4e1adf3db1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:56.235477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1259/2514] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:29:56 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:56.239893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a7192-9360-468a-8556-f2732181de37 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:56.249774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a7192-9360-468a-8556-f2732181de37 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:56.309557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68799148-7332-480e-b1b5-60982156cda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:56.310151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68799148-7332-480e-b1b5-60982156cda2 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.310430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68799148-7332-480e-b1b5-60982156cda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.310699 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68799148-7332-480e-b1b5-60982156cda2 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.311324 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68799148-7332-480e-b1b5-60982156cda2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:56.313299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68799148-7332-480e-b1b5-60982156cda2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:56.335376 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1256/2515] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:29:56 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:29:56.339585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5db53-bbcf-4b44-b9eb-1a4e1adf3db1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:56.350512 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5db53-bbcf-4b44-b9eb-1a4e1adf3db1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:56.355137 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee607-1e75-441c-b293-ceb69b50470a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:56.355734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee607-1e75-441c-b293-ceb69b50470a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.356013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee607-1e75-441c-b293-ceb69b50470a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.356284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee607-1e75-441c-b293-ceb69b50470a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.356953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ee607-1e75-441c-b293-ceb69b50470a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:56.358641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ee607-1e75-441c-b293-ceb69b50470a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:56.368483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1260/2516] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:29:56.372336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68799148-7332-480e-b1b5-60982156cda2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:56.382835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68799148-7332-480e-b1b5-60982156cda2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:56.387617 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900be6b-ac66-4ad3-80df-b6e52f532d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:56.388251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900be6b-ac66-4ad3-80df-b6e52f532d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.388568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900be6b-ac66-4ad3-80df-b6e52f532d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.388835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900be6b-ac66-4ad3-80df-b6e52f532d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.389471 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900be6b-ac66-4ad3-80df-b6e52f532d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:56.391166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900be6b-ac66-4ad3-80df-b6e52f532d73 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:56.402432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1257/2517] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:29:56.406534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee607-1e75-441c-b293-ceb69b50470a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:56.416775 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee607-1e75-441c-b293-ceb69b50470a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:56.421430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b521-ba49-465a-a0fc-c9b93c900756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:56.422069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b521-ba49-465a-a0fc-c9b93c900756 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.422350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b521-ba49-465a-a0fc-c9b93c900756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.422620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b521-ba49-465a-a0fc-c9b93c900756 admin admin] PATH_INFO: `/v3/projects/05932e06378e4003971ac1b659107747/users/86c0d10820ed45b9960704a6c4de2b6d/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.438295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8b521-ba49-465a-a0fc-c9b93c900756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05932e06378e4003971ac1b659107747, user_id=86c0d10820ed45b9960704a6c4de2b6d, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:56.441516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8b521-ba49-465a-a0fc-c9b93c900756 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:56.454455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1261/2518] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:29:56 2022] PUT /identity/v3/projects/05932e06378e4003971ac1b659107747/users/86c0d10820ed45b9960704a6c4de2b6d/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:29:56.466162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1fc0a-b1eb-4de0-9cc0-079069901314 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.466451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1fc0a-b1eb-4de0-9cc0-079069901314 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.466728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1fc0a-b1eb-4de0-9cc0-079069901314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.494340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-90d1fc0a-b1eb-4de0-9cc0-079069901314 None None] MFA Rules not processed for user `86c0d10820ed45b9960704a6c4de2b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:29:56.495543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-90d1fc0a-b1eb-4de0-9cc0-079069901314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:56.559607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1258/2519] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:29:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:29:56.571946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b521-ba49-465a-a0fc-c9b93c900756 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:29:56.573724 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dcb8b521-ba49-465a-a0fc-c9b93c900756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:29:56.629793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b521-ba49-465a-a0fc-c9b93c900756 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:29:56.633464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ea87c-dc46-46c6-820f-9445f0358cb9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0dfcbad22348fdaea32d64557e68a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:29:56.634067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ea87c-dc46-46c6-820f-9445f0358cb9 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:29:56.634362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ea87c-dc46-46c6-820f-9445f0358cb9 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:29:56.634621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ea87c-dc46-46c6-820f-9445f0358cb9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:29:56.642971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ea87c-dc46-46c6-820f-9445f0358cb9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:29:56.644763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ea87c-dc46-46c6-820f-9445f0358cb9 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:29:56.653538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1262/2520] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:29:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:06.556743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:06.649628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:08.735382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1fc0a-b1eb-4de0-9cc0-079069901314 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:08.736922 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-90d1fc0a-b1eb-4de0-9cc0-079069901314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:08.806648 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1fc0a-b1eb-4de0-9cc0-079069901314 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:08.810290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1598ed-b46c-4f2c-bfa5-0b6358b01e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:08.810889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1598ed-b46c-4f2c-bfa5-0b6358b01e22 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:08.811181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1598ed-b46c-4f2c-bfa5-0b6358b01e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:08.811465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1598ed-b46c-4f2c-bfa5-0b6358b01e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:08.813517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cc1598ed-b46c-4f2c-bfa5-0b6358b01e22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:08.883383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1598ed-b46c-4f2c-bfa5-0b6358b01e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:08.885043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1598ed-b46c-4f2c-bfa5-0b6358b01e22 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:08.902824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1259/2521] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:12.396770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ea87c-dc46-46c6-820f-9445f0358cb9 service glance] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:12.399198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-744ea87c-dc46-46c6-820f-9445f0358cb9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:12.419035 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ea87c-dc46-46c6-820f-9445f0358cb9 service glance] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:12.423685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:12.424314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:12.424647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:12.424926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] PATH_INFO: `/v3/users/4aaa94a183fb49b79e6383ea783ab76d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:12.426388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aaa94a183fb49b79e6383ea783ab76d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:12.428477 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:12.483821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aaa94a183fb49b79e6383ea783ab76d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.484874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aaa94a183fb49b79e6383ea783ab76d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.485753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aaa94a183fb49b79e6383ea783ab76d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.498708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aaa94a183fb49b79e6383ea783ab76d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.513316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-23609190-c383-4a8e-80a7-d4c963434b1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aaa94a183fb49b79e6383ea783ab76d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.531368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1263/2522] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:12 2022] DELETE /identity/v3/users/4aaa94a183fb49b79e6383ea783ab76d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:12.760217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1598ed-b46c-4f2c-bfa5-0b6358b01e22 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:12.770763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1598ed-b46c-4f2c-bfa5-0b6358b01e22 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:12.831252 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:12.831885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:12.832906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:12.833192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] PATH_INFO: `/v3/projects/cbf14c554d854af5ba8cd910c82d42e0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:12.834398 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf14c554d854af5ba8cd910c82d42e0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:12.836533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:12.883206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf14c554d854af5ba8cd910c82d42e0', 'request_id': 'req-2b99d60d-88bf-417b-bd36-f61e42241554'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.885198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:12.890189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf14c554d854af5ba8cd910c82d42e0', 'request_id': 'req-2b99d60d-88bf-417b-bd36-f61e42241554'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.898193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf14c554d854af5ba8cd910c82d42e0', 'request_id': 'req-2b99d60d-88bf-417b-bd36-f61e42241554'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.909689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] The token cache is being invalidate because project cbf14c554d854af5ba8cd910c82d42e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:12.909976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.911391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2b99d60d-88bf-417b-bd36-f61e42241554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:12.913394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1260/2523] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:12 2022] DELETE /identity/v3/projects/cbf14c554d854af5ba8cd910c82d42e0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:12.918322 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9ac0f-6a10-4438-9520-f48d53ba83db None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:12.918605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9ac0f-6a10-4438-9520-f48d53ba83db None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:12.918887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9ac0f-6a10-4438-9520-f48d53ba83db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:12.942646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-67c9ac0f-6a10-4438-9520-f48d53ba83db None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:12.944005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-67c9ac0f-6a10-4438-9520-f48d53ba83db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:13.015137 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1264/2524] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:13.016610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128aa26-8856-493c-980d-21f7d547e723 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:13.016900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128aa26-8856-493c-980d-21f7d547e723 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:13.017745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128aa26-8856-493c-980d-21f7d547e723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:13.038660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5128aa26-8856-493c-980d-21f7d547e723 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:13.039943 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5128aa26-8856-493c-980d-21f7d547e723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:13.050986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1261/2525] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:13.055287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9ac0f-6a10-4438-9520-f48d53ba83db None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:13.065794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9ac0f-6a10-4438-9520-f48d53ba83db None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:13.070873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7f22a-3d00-4a7d-a8ee-98a3b3ee7169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:13.071495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7f22a-3d00-4a7d-a8ee-98a3b3ee7169 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:13.071802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7f22a-3d00-4a7d-a8ee-98a3b3ee7169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:13.072057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7f22a-3d00-4a7d-a8ee-98a3b3ee7169 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:13.072768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7f22a-3d00-4a7d-a8ee-98a3b3ee7169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:30:13.073077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7f22a-3d00-4a7d-a8ee-98a3b3ee7169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:13.075114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7f22a-3d00-4a7d-a8ee-98a3b3ee7169 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:13.090368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1265/2526] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:30:13.101448 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128aa26-8856-493c-980d-21f7d547e723 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:13.111703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128aa26-8856-493c-980d-21f7d547e723 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:13.117202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb0507-0222-4d8f-ae4e-37e8ae351014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:13.117838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb0507-0222-4d8f-ae4e-37e8ae351014 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:13.118123 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb0507-0222-4d8f-ae4e-37e8ae351014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:13.118396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb0507-0222-4d8f-ae4e-37e8ae351014 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:13.119171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb0507-0222-4d8f-ae4e-37e8ae351014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:13.121759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb0507-0222-4d8f-ae4e-37e8ae351014 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:13.157062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1262/2527] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:13 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:13.161583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7f22a-3d00-4a7d-a8ee-98a3b3ee7169 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:13.180142 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7f22a-3d00-4a7d-a8ee-98a3b3ee7169 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:13.250923 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701aa8b-b5ad-4d7f-bf48-e816dde88bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:13.251625 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701aa8b-b5ad-4d7f-bf48-e816dde88bae admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:13.251959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701aa8b-b5ad-4d7f-bf48-e816dde88bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:13.252435 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701aa8b-b5ad-4d7f-bf48-e816dde88bae admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:13.253178 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701aa8b-b5ad-4d7f-bf48-e816dde88bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:13.255425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701aa8b-b5ad-4d7f-bf48-e816dde88bae admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:13.268687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:13.290730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1266/2528] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:13 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:13.295235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb0507-0222-4d8f-ae4e-37e8ae351014 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:13.308814 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb0507-0222-4d8f-ae4e-37e8ae351014 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:13.317345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca738-18d0-4b74-bf6a-bad049cd6470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:13.317927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca738-18d0-4b74-bf6a-bad049cd6470 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:13.318394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca738-18d0-4b74-bf6a-bad049cd6470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:13.318678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca738-18d0-4b74-bf6a-bad049cd6470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:13.319503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ca738-18d0-4b74-bf6a-bad049cd6470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:13.321377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ca738-18d0-4b74-bf6a-bad049cd6470 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:13.336308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1263/2529] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:13.338213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701aa8b-b5ad-4d7f-bf48-e816dde88bae admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:13.352002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701aa8b-b5ad-4d7f-bf48-e816dde88bae admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:13.357730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0f150-3d8e-46c6-a3c3-92d63d8de5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:13.358347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0f150-3d8e-46c6-a3c3-92d63d8de5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:13.358888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0f150-3d8e-46c6-a3c3-92d63d8de5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:13.359161 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0f150-3d8e-46c6-a3c3-92d63d8de5f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:13.359845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0f150-3d8e-46c6-a3c3-92d63d8de5f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:13.361900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0f150-3d8e-46c6-a3c3-92d63d8de5f9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:13.373891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1267/2530] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:13.377940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca738-18d0-4b74-bf6a-bad049cd6470 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:13.393489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca738-18d0-4b74-bf6a-bad049cd6470 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:13.403918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f931a-6fad-408e-a4ef-2c22422c8e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:13.405798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f931a-6fad-408e-a4ef-2c22422c8e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:13.405798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f931a-6fad-408e-a4ef-2c22422c8e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:13.406030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f931a-6fad-408e-a4ef-2c22422c8e27 admin admin] PATH_INFO: `/v3/projects/213ae78186aa4ef582e8279302165d76/users/28148ef7eeb94dbe9dcc11759e9daf50/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:13.430422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f931a-6fad-408e-a4ef-2c22422c8e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213ae78186aa4ef582e8279302165d76, user_id=28148ef7eeb94dbe9dcc11759e9daf50, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:13.433523 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f931a-6fad-408e-a4ef-2c22422c8e27 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:13.449030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1264/2531] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:13 2022] PUT /identity/v3/projects/213ae78186aa4ef582e8279302165d76/users/28148ef7eeb94dbe9dcc11759e9daf50/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:13.460087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7786e4-b3b4-44d1-842e-fb229e2b8245 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:13.460615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7786e4-b3b4-44d1-842e-fb229e2b8245 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:13.460711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7786e4-b3b4-44d1-842e-fb229e2b8245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:13.491806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3b7786e4-b3b4-44d1-842e-fb229e2b8245 None None] MFA Rules not processed for user `28148ef7eeb94dbe9dcc11759e9daf50`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:13.493058 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b7786e4-b3b4-44d1-842e-fb229e2b8245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:13.575367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1268/2532] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:13.587576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f931a-6fad-408e-a4ef-2c22422c8e27 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:13.589418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e21f931a-6fad-408e-a4ef-2c22422c8e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:13.665974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f931a-6fad-408e-a4ef-2c22422c8e27 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:13.671582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bb14d-c7d2-4c1a-a730-f460ebcd21fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:13.672186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bb14d-c7d2-4c1a-a730-f460ebcd21fa service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:13.672495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bb14d-c7d2-4c1a-a730-f460ebcd21fa service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:13.672766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bb14d-c7d2-4c1a-a730-f460ebcd21fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:13.681806 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bb14d-c7d2-4c1a-a730-f460ebcd21fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:13.683392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bb14d-c7d2-4c1a-a730-f460ebcd21fa service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:13.693947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1265/2533] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:19.962901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7786e4-b3b4-44d1-842e-fb229e2b8245 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:20.024644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7786e4-b3b4-44d1-842e-fb229e2b8245 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:20.029295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:20.029921 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:20.030207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:20.030483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] PATH_INFO: `/v3/users/28148ef7eeb94dbe9dcc11759e9daf50` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:20.031617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28148ef7eeb94dbe9dcc11759e9daf50)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:20.033603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:20.081117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28148ef7eeb94dbe9dcc11759e9daf50'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.081804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28148ef7eeb94dbe9dcc11759e9daf50'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.083074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28148ef7eeb94dbe9dcc11759e9daf50'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.090399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28148ef7eeb94dbe9dcc11759e9daf50'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.100768 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:20.103311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28148ef7eeb94dbe9dcc11759e9daf50'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.121810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1269/2534] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:19 2022] DELETE /identity/v3/users/28148ef7eeb94dbe9dcc11759e9daf50 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:20.126256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4489271-a152-445f-b977-559d36445138 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:20.126705 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4489271-a152-445f-b977-559d36445138 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:20.126810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4489271-a152-445f-b977-559d36445138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:20.152664 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c4489271-a152-445f-b977-559d36445138 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:20.154623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c4489271-a152-445f-b977-559d36445138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:20.236207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1266/2535] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:20.245204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:20.246710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:20.338732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e6821-d1e2-4bbb-af1f-05548c69b85f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:20.345182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e2684-ae02-41ac-bf66-9bed3b8e6f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:20.345182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e2684-ae02-41ac-bf66-9bed3b8e6f6d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:20.345182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e2684-ae02-41ac-bf66-9bed3b8e6f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:20.345182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e2684-ae02-41ac-bf66-9bed3b8e6f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:20.357175 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e2684-ae02-41ac-bf66-9bed3b8e6f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:20.359138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e2684-ae02-41ac-bf66-9bed3b8e6f6d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:20.373045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1270/2536] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:20.679282 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4489271-a152-445f-b977-559d36445138 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:20.680898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c4489271-a152-445f-b977-559d36445138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:20.689851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:20.698250 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4489271-a152-445f-b977-559d36445138 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:20.704696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:20.705396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:20.705712 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:20.705985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] PATH_INFO: `/v3/projects/213ae78186aa4ef582e8279302165d76` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:20.708094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213ae78186aa4ef582e8279302165d76)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:20.711022 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:20.758257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213ae78186aa4ef582e8279302165d76', 'request_id': 'req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.763711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213ae78186aa4ef582e8279302165d76', 'request_id': 'req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.770723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213ae78186aa4ef582e8279302165d76', 'request_id': 'req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.781638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] The token cache is being invalidate because project 213ae78186aa4ef582e8279302165d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:20.781921 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.782556 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e2a931d5-f0ee-4481-9a03-e1ebc532ae70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:20.784021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1267/2537] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:20 2022] DELETE /identity/v3/projects/213ae78186aa4ef582e8279302165d76 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:20.789438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eaa10-ca09-41c1-91d6-502495124f46 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:20.789774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eaa10-ca09-41c1-91d6-502495124f46 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:20.790047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eaa10-ca09-41c1-91d6-502495124f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:20.811168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-070eaa10-ca09-41c1-91d6-502495124f46 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:20.812476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-070eaa10-ca09-41c1-91d6-502495124f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:20.872848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1271/2538] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:20.882130 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa86d0-6b91-4989-bd39-23eba3347553 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:20.882419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa86d0-6b91-4989-bd39-23eba3347553 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:20.882703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa86d0-6b91-4989-bd39-23eba3347553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:20.902181 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-17fa86d0-6b91-4989-bd39-23eba3347553 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:20.903383 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-17fa86d0-6b91-4989-bd39-23eba3347553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:20.911866 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1268/2539] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:20.916038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eaa10-ca09-41c1-91d6-502495124f46 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:20.926749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eaa10-ca09-41c1-91d6-502495124f46 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:20.931303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19458-3c20-40c5-a4fb-8a00c2251286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:20.931893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19458-3c20-40c5-a4fb-8a00c2251286 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:20.932170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19458-3c20-40c5-a4fb-8a00c2251286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:20.932469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19458-3c20-40c5-a4fb-8a00c2251286 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:20.933106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19458-3c20-40c5-a4fb-8a00c2251286 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:30:20.933410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19458-3c20-40c5-a4fb-8a00c2251286 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:20.935024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19458-3c20-40c5-a4fb-8a00c2251286 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:20.949134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1272/2540] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:30:20.953457 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa86d0-6b91-4989-bd39-23eba3347553 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:20.963947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa86d0-6b91-4989-bd39-23eba3347553 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:20.968808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db742d85-ad15-44dd-aaaf-538e44d609ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:20.969604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db742d85-ad15-44dd-aaaf-538e44d609ba admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:20.970003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db742d85-ad15-44dd-aaaf-538e44d609ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:20.970569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db742d85-ad15-44dd-aaaf-538e44d609ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:20.971313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db742d85-ad15-44dd-aaaf-538e44d609ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:20.973982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db742d85-ad15-44dd-aaaf-538e44d609ba admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.007122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1269/2541] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:20 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:21.008603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19458-3c20-40c5-a4fb-8a00c2251286 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.026796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19458-3c20-40c5-a4fb-8a00c2251286 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.086004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a9851-1e5f-49cf-9e00-5424f0cecc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.086612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a9851-1e5f-49cf-9e00-5424f0cecc53 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.086888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a9851-1e5f-49cf-9e00-5424f0cecc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.087153 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a9851-1e5f-49cf-9e00-5424f0cecc53 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.087779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a9851-1e5f-49cf-9e00-5424f0cecc53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.089726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a9851-1e5f-49cf-9e00-5424f0cecc53 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.112449 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1273/2542] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:21 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:21.117487 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db742d85-ad15-44dd-aaaf-538e44d609ba admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.128730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db742d85-ad15-44dd-aaaf-538e44d609ba admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.133596 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ad4a-a66e-4aab-a517-8d60d6e594dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.134197 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ad4a-a66e-4aab-a517-8d60d6e594dd admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.134497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ad4a-a66e-4aab-a517-8d60d6e594dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.134774 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0ad4a-a66e-4aab-a517-8d60d6e594dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.135418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0ad4a-a66e-4aab-a517-8d60d6e594dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.137058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0ad4a-a66e-4aab-a517-8d60d6e594dd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.147209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1270/2543] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:21.151145 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a9851-1e5f-49cf-9e00-5424f0cecc53 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.161863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a9851-1e5f-49cf-9e00-5424f0cecc53 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.166544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde7175-6ebc-474d-88d1-2994192b901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.167139 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde7175-6ebc-474d-88d1-2994192b901a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.167423 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde7175-6ebc-474d-88d1-2994192b901a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.167701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde7175-6ebc-474d-88d1-2994192b901a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.168350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde7175-6ebc-474d-88d1-2994192b901a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.170024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde7175-6ebc-474d-88d1-2994192b901a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.171412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:21.186120 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1274/2544] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:21.187293 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ad4a-a66e-4aab-a517-8d60d6e594dd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.197262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0ad4a-a66e-4aab-a517-8d60d6e594dd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.202062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72631d-cf50-443b-b173-a1bddf287395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.202698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72631d-cf50-443b-b173-a1bddf287395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.202977 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72631d-cf50-443b-b173-a1bddf287395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.203256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72631d-cf50-443b-b173-a1bddf287395 admin admin] PATH_INFO: `/v3/projects/963e5018967242b2bd43c9b8598582bb/users/d96e34edb92f485696d5ea8ab2024496/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.218267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72631d-cf50-443b-b173-a1bddf287395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=963e5018967242b2bd43c9b8598582bb, user_id=d96e34edb92f485696d5ea8ab2024496, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.220975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72631d-cf50-443b-b173-a1bddf287395 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.234407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1271/2545] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:21 2022] PUT /identity/v3/projects/963e5018967242b2bd43c9b8598582bb/users/d96e34edb92f485696d5ea8ab2024496/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:21.244642 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e8125-71be-4280-a9a7-43d21ade5949 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.244949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e8125-71be-4280-a9a7-43d21ade5949 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.245224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e8125-71be-4280-a9a7-43d21ade5949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.272896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3c9e8125-71be-4280-a9a7-43d21ade5949 None None] MFA Rules not processed for user `d96e34edb92f485696d5ea8ab2024496`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:21.274097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3c9e8125-71be-4280-a9a7-43d21ade5949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:21.337876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1275/2546] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:21.342396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72631d-cf50-443b-b173-a1bddf287395 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.404551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72631d-cf50-443b-b173-a1bddf287395 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.409203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1e044-3264-4f3b-8595-3f4be0271086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.409815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1e044-3264-4f3b-8595-3f4be0271086 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.410096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1e044-3264-4f3b-8595-3f4be0271086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.410368 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1e044-3264-4f3b-8595-3f4be0271086 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.412026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1e044-3264-4f3b-8595-3f4be0271086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.413674 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1e044-3264-4f3b-8595-3f4be0271086 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.444047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1272/2547] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:21 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:21.445192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e8125-71be-4280-a9a7-43d21ade5949 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.456411 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e8125-71be-4280-a9a7-43d21ade5949 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.518210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bb288-2910-43d8-924c-5a12e37c3bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.519202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bb288-2910-43d8-924c-5a12e37c3bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.519644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bb288-2910-43d8-924c-5a12e37c3bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.521202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bb288-2910-43d8-924c-5a12e37c3bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.522026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bb288-2910-43d8-924c-5a12e37c3bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.524663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bb288-2910-43d8-924c-5a12e37c3bb5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.549984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1276/2548] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:21 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:30:21.554636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1e044-3264-4f3b-8595-3f4be0271086 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.566207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1e044-3264-4f3b-8595-3f4be0271086 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.572765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d346c-9ac0-4378-91ad-51fcddadc316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.573466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d346c-9ac0-4378-91ad-51fcddadc316 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.573752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d346c-9ac0-4378-91ad-51fcddadc316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.574036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d346c-9ac0-4378-91ad-51fcddadc316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.574683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d346c-9ac0-4378-91ad-51fcddadc316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.576384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d346c-9ac0-4378-91ad-51fcddadc316 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.577895 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:21.590982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1273/2549] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:21.594681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bb288-2910-43d8-924c-5a12e37c3bb5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.606566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bb288-2910-43d8-924c-5a12e37c3bb5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.611771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef258-4631-4215-a7a8-0d9c400645fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.612497 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef258-4631-4215-a7a8-0d9c400645fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.612780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef258-4631-4215-a7a8-0d9c400645fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.613059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef258-4631-4215-a7a8-0d9c400645fe admin admin] PATH_INFO: `/v3/projects/a8c3ac3320914a8f963fabddeb257975/users/4fbc5050a924445eac6b9ef696c067ea/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.628668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef258-4631-4215-a7a8-0d9c400645fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c3ac3320914a8f963fabddeb257975, user_id=4fbc5050a924445eac6b9ef696c067ea, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.631929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef258-4631-4215-a7a8-0d9c400645fe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.646446 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1277/2550] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:21 2022] PUT /identity/v3/projects/a8c3ac3320914a8f963fabddeb257975/users/4fbc5050a924445eac6b9ef696c067ea/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:21.650521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d346c-9ac0-4378-91ad-51fcddadc316 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.660718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d346c-9ac0-4378-91ad-51fcddadc316 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.721966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f86aa9-e3ed-4a93-86b2-6a05ec2c8664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.722958 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f86aa9-e3ed-4a93-86b2-6a05ec2c8664 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.723355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f86aa9-e3ed-4a93-86b2-6a05ec2c8664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.723635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f86aa9-e3ed-4a93-86b2-6a05ec2c8664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.724294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f86aa9-e3ed-4a93-86b2-6a05ec2c8664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.725942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f86aa9-e3ed-4a93-86b2-6a05ec2c8664 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.737898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1274/2551] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:21.741926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef258-4631-4215-a7a8-0d9c400645fe admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.752680 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef258-4631-4215-a7a8-0d9c400645fe admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.758125 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59f128-9377-491d-92b1-eb13bf086afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.761958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59f128-9377-491d-92b1-eb13bf086afc admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.762246 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59f128-9377-491d-92b1-eb13bf086afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.762555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59f128-9377-491d-92b1-eb13bf086afc admin admin] PATH_INFO: `/v3/projects/a8c3ac3320914a8f963fabddeb257975/users/4fbc5050a924445eac6b9ef696c067ea/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.764639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59f128-9377-491d-92b1-eb13bf086afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c3ac3320914a8f963fabddeb257975, user_id=4fbc5050a924445eac6b9ef696c067ea, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.767699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59f128-9377-491d-92b1-eb13bf086afc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.778234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1278/2552] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:21 2022] PUT /identity/v3/projects/a8c3ac3320914a8f963fabddeb257975/users/4fbc5050a924445eac6b9ef696c067ea/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:21.789169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708831-0104-4ef0-b4f5-d688b479c6e6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.789464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708831-0104-4ef0-b4f5-d688b479c6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.789747 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708831-0104-4ef0-b4f5-d688b479c6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.820092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-db708831-0104-4ef0-b4f5-d688b479c6e6 None None] MFA Rules not processed for user `4fbc5050a924445eac6b9ef696c067ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:21.821375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-db708831-0104-4ef0-b4f5-d688b479c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:21.896234 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1275/2553] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:21.907146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59f128-9377-491d-92b1-eb13bf086afc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:21.908992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e59f128-9377-491d-92b1-eb13bf086afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:21.952349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:21.968432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59f128-9377-491d-92b1-eb13bf086afc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:21.972072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f590968-dce1-4cd3-91d6-00c0018678f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:21.972701 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f590968-dce1-4cd3-91d6-00c0018678f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:21.973006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f590968-dce1-4cd3-91d6-00c0018678f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:21.973278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f590968-dce1-4cd3-91d6-00c0018678f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:21.982172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f590968-dce1-4cd3-91d6-00c0018678f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:21.983617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f590968-dce1-4cd3-91d6-00c0018678f8 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:21.993787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1279/2554] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:22.390380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db708831-0104-4ef0-b4f5-d688b479c6e6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:22.455351 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db708831-0104-4ef0-b4f5-d688b479c6e6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:22.460047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:22.460734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:22.461020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:22.461294 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] PATH_INFO: `/v3/users/d96e34edb92f485696d5ea8ab2024496` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:22.462427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96e34edb92f485696d5ea8ab2024496)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:22.464354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:22.485792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:22.511960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96e34edb92f485696d5ea8ab2024496'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:22.518856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96e34edb92f485696d5ea8ab2024496'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:22.519507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96e34edb92f485696d5ea8ab2024496'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:22.531466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96e34edb92f485696d5ea8ab2024496'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:22.539347 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96e34edb92f485696d5ea8ab2024496'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:22.545368 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1276/2555] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:22 2022] DELETE /identity/v3/users/d96e34edb92f485696d5ea8ab2024496 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:22.549869 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bccea-d19f-42c0-adcf-b15649004df1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:22.550361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bccea-d19f-42c0-adcf-b15649004df1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:22.550455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bccea-d19f-42c0-adcf-b15649004df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:22.571257 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-436bccea-d19f-42c0-adcf-b15649004df1 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:22.572447 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-436bccea-d19f-42c0-adcf-b15649004df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:22.634143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1280/2556] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:22.643751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:22.646025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:22.705708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50a21f-50d0-44c5-bd4d-77b93c8770c8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:22.709372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c52b5d-8d7e-4e31-ac88-b98f40ee643c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:22.709987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c52b5d-8d7e-4e31-ac88-b98f40ee643c service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:22.710260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c52b5d-8d7e-4e31-ac88-b98f40ee643c service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:22.710536 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c52b5d-8d7e-4e31-ac88-b98f40ee643c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:22.719339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c52b5d-8d7e-4e31-ac88-b98f40ee643c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:22.720819 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c52b5d-8d7e-4e31-ac88-b98f40ee643c service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:22.730502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1277/2557] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:23.049401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bccea-d19f-42c0-adcf-b15649004df1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:23.051146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-436bccea-d19f-42c0-adcf-b15649004df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:23.064029 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bccea-d19f-42c0-adcf-b15649004df1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:23.069466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:23.070147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:23.070470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:23.070747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] PATH_INFO: `/v3/projects/963e5018967242b2bd43c9b8598582bb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:23.072105 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=963e5018967242b2bd43c9b8598582bb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:23.074072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:23.123113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963e5018967242b2bd43c9b8598582bb', 'request_id': 'req-4a03e563-36e7-469d-9342-1fc63024d7c5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.128713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963e5018967242b2bd43c9b8598582bb', 'request_id': 'req-4a03e563-36e7-469d-9342-1fc63024d7c5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.136784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963e5018967242b2bd43c9b8598582bb', 'request_id': 'req-4a03e563-36e7-469d-9342-1fc63024d7c5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.148942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] The token cache is being invalidate because project 963e5018967242b2bd43c9b8598582bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:23.149233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.149861 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.151352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1281/2558] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:23 2022] DELETE /identity/v3/projects/963e5018967242b2bd43c9b8598582bb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:23.155366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c52b5d-8d7e-4e31-ac88-b98f40ee643c service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:23.165733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c52b5d-8d7e-4e31-ac88-b98f40ee643c service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:23.221996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:23.222548 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:23.222829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:23.223650 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] PATH_INFO: `/v3/users/4fbc5050a924445eac6b9ef696c067ea` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:23.224896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fbc5050a924445eac6b9ef696c067ea)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:23.226842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:23.266458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:23.272110 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fbc5050a924445eac6b9ef696c067ea'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.280253 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fbc5050a924445eac6b9ef696c067ea'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.280945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fbc5050a924445eac6b9ef696c067ea'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.293162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fbc5050a924445eac6b9ef696c067ea'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.300509 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-44739691-5b78-4dc5-b974-5c5d2a5efb6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fbc5050a924445eac6b9ef696c067ea'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.307188 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1278/2559] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:23 2022] DELETE /identity/v3/users/4fbc5050a924445eac6b9ef696c067ea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:23.672067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:23.680816 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:23.696514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03e563-36e7-469d-9342-1fc63024d7c5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:23.756622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:23.757257 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:23.757537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:23.757810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] PATH_INFO: `/v3/projects/a8c3ac3320914a8f963fabddeb257975` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:23.758928 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c3ac3320914a8f963fabddeb257975)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:23.760918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:23.767164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:23.808971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c3ac3320914a8f963fabddeb257975', 'request_id': 'req-2888759a-2a52-4c11-bca2-85694e0f47e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.814854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c3ac3320914a8f963fabddeb257975', 'request_id': 'req-2888759a-2a52-4c11-bca2-85694e0f47e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.825496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c3ac3320914a8f963fabddeb257975', 'request_id': 'req-2888759a-2a52-4c11-bca2-85694e0f47e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.838279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] The token cache is being invalidate because project a8c3ac3320914a8f963fabddeb257975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:23.838567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.839222 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2888759a-2a52-4c11-bca2-85694e0f47e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:23.840730 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1282/2560] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:23 2022] DELETE /identity/v3/projects/a8c3ac3320914a8f963fabddeb257975 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:23.845525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d5c52-f565-4e4e-9ea5-4a6fb75a9991 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:23.845994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d5c52-f565-4e4e-9ea5-4a6fb75a9991 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:23.846090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d5c52-f565-4e4e-9ea5-4a6fb75a9991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:23.867572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e40d5c52-f565-4e4e-9ea5-4a6fb75a9991 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:23.868820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e40d5c52-f565-4e4e-9ea5-4a6fb75a9991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:23.927888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1279/2561] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:23.937094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee70e82-436a-44cb-870f-051851aaf6d9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:23.937377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee70e82-436a-44cb-870f-051851aaf6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:23.937648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee70e82-436a-44cb-870f-051851aaf6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:23.957459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cee70e82-436a-44cb-870f-051851aaf6d9 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:23.958953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cee70e82-436a-44cb-870f-051851aaf6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:23.966760 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1283/2562] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:23.970939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d5c52-f565-4e4e-9ea5-4a6fb75a9991 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:23.981187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d5c52-f565-4e4e-9ea5-4a6fb75a9991 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:23.985843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ffaf8-15b2-43c3-910f-a2fb982cca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:23.986266 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ffaf8-15b2-43c3-910f-a2fb982cca24 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:23.986545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ffaf8-15b2-43c3-910f-a2fb982cca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:23.986816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ffaf8-15b2-43c3-910f-a2fb982cca24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:23.987460 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ffaf8-15b2-43c3-910f-a2fb982cca24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:30:23.987757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ffaf8-15b2-43c3-910f-a2fb982cca24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:23.990095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ffaf8-15b2-43c3-910f-a2fb982cca24 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.002913 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1280/2563] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:30:24.008224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee70e82-436a-44cb-870f-051851aaf6d9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.018570 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee70e82-436a-44cb-870f-051851aaf6d9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.023503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4b8e-ec8a-476f-bc4a-68fb511a2251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.024265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4b8e-ec8a-476f-bc4a-68fb511a2251 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.024830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4b8e-ec8a-476f-bc4a-68fb511a2251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.025107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4b8e-ec8a-476f-bc4a-68fb511a2251 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.026019 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b4b8e-ec8a-476f-bc4a-68fb511a2251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.028254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b4b8e-ec8a-476f-bc4a-68fb511a2251 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.065169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1284/2564] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:24 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:24.066272 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ffaf8-15b2-43c3-910f-a2fb982cca24 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.077422 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ffaf8-15b2-43c3-910f-a2fb982cca24 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.139249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562e3d7-eb48-4d8f-8b59-a57371008dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.139889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562e3d7-eb48-4d8f-8b59-a57371008dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.140193 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562e3d7-eb48-4d8f-8b59-a57371008dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.140499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562e3d7-eb48-4d8f-8b59-a57371008dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.141127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562e3d7-eb48-4d8f-8b59-a57371008dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.143042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562e3d7-eb48-4d8f-8b59-a57371008dd6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.169065 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1281/2565] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:24 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:30:24.170279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4b8e-ec8a-476f-bc4a-68fb511a2251 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.183085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4b8e-ec8a-476f-bc4a-68fb511a2251 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.188704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9e067-4fb7-4f61-aeba-c67f47d9679b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.189748 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9e067-4fb7-4f61-aeba-c67f47d9679b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.190213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9e067-4fb7-4f61-aeba-c67f47d9679b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.190496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9e067-4fb7-4f61-aeba-c67f47d9679b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.191149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9e067-4fb7-4f61-aeba-c67f47d9679b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.193418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9e067-4fb7-4f61-aeba-c67f47d9679b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.203686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1285/2566] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:24.208966 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562e3d7-eb48-4d8f-8b59-a57371008dd6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.219726 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562e3d7-eb48-4d8f-8b59-a57371008dd6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.226272 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f4c50-84b1-4989-a1c0-33a65e266da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.226861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f4c50-84b1-4989-a1c0-33a65e266da3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.227164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f4c50-84b1-4989-a1c0-33a65e266da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.227433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f4c50-84b1-4989-a1c0-33a65e266da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.228162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f4c50-84b1-4989-a1c0-33a65e266da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.230320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f4c50-84b1-4989-a1c0-33a65e266da3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.241541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1282/2567] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:24.245946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9e067-4fb7-4f61-aeba-c67f47d9679b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.256226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9e067-4fb7-4f61-aeba-c67f47d9679b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.260970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41860e7e-29d4-4e80-88fd-7738ccd8f378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.261613 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41860e7e-29d4-4e80-88fd-7738ccd8f378 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.261891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41860e7e-29d4-4e80-88fd-7738ccd8f378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.262162 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41860e7e-29d4-4e80-88fd-7738ccd8f378 admin admin] PATH_INFO: `/v3/projects/22f6cef1460e465fa102cecbd3904283/users/c6edfcff5efc4434ba0c6f50271cd7c5/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.278079 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41860e7e-29d4-4e80-88fd-7738ccd8f378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f6cef1460e465fa102cecbd3904283, user_id=c6edfcff5efc4434ba0c6f50271cd7c5, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.280808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41860e7e-29d4-4e80-88fd-7738ccd8f378 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.294435 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1286/2568] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:24 2022] PUT /identity/v3/projects/22f6cef1460e465fa102cecbd3904283/users/c6edfcff5efc4434ba0c6f50271cd7c5/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:24.305097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f4aa4-5dd1-4a47-9579-bfe0789e47d0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.305381 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f4aa4-5dd1-4a47-9579-bfe0789e47d0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.305677 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f4aa4-5dd1-4a47-9579-bfe0789e47d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.334549 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-719f4aa4-5dd1-4a47-9579-bfe0789e47d0 None None] MFA Rules not processed for user `c6edfcff5efc4434ba0c6f50271cd7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:24.335704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-719f4aa4-5dd1-4a47-9579-bfe0789e47d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:24.348807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:24.402048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1283/2569] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:24.406764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41860e7e-29d4-4e80-88fd-7738ccd8f378 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.468451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41860e7e-29d4-4e80-88fd-7738ccd8f378 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.473766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518e706-3a2d-4241-8833-0a97491c717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.474367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518e706-3a2d-4241-8833-0a97491c717b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.474648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518e706-3a2d-4241-8833-0a97491c717b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.474919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518e706-3a2d-4241-8833-0a97491c717b admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.475542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518e706-3a2d-4241-8833-0a97491c717b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.477508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518e706-3a2d-4241-8833-0a97491c717b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.503424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1287/2570] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:24 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:24.507591 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f4aa4-5dd1-4a47-9579-bfe0789e47d0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.518791 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f4aa4-5dd1-4a47-9579-bfe0789e47d0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.575176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e25c65-dd3a-4b3d-a5cb-e05040a88e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.575784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e25c65-dd3a-4b3d-a5cb-e05040a88e7c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.576067 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e25c65-dd3a-4b3d-a5cb-e05040a88e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.576345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e25c65-dd3a-4b3d-a5cb-e05040a88e7c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.577012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e25c65-dd3a-4b3d-a5cb-e05040a88e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.579059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e25c65-dd3a-4b3d-a5cb-e05040a88e7c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.602361 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1284/2571] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:24 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:24.606104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518e706-3a2d-4241-8833-0a97491c717b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.616878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518e706-3a2d-4241-8833-0a97491c717b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.621609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1916-5026-4a08-b2f9-8e81347498b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.622235 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1916-5026-4a08-b2f9-8e81347498b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.622520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1916-5026-4a08-b2f9-8e81347498b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.622794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1916-5026-4a08-b2f9-8e81347498b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.623462 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1916-5026-4a08-b2f9-8e81347498b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.625141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1916-5026-4a08-b2f9-8e81347498b6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.641463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1288/2572] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:24.642675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e25c65-dd3a-4b3d-a5cb-e05040a88e7c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.653972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e25c65-dd3a-4b3d-a5cb-e05040a88e7c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.658682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b3c9f-ac0d-4d87-af6f-22d39fedf910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.659331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b3c9f-ac0d-4d87-af6f-22d39fedf910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.659612 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b3c9f-ac0d-4d87-af6f-22d39fedf910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.659889 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b3c9f-ac0d-4d87-af6f-22d39fedf910 admin admin] PATH_INFO: `/v3/projects/41193db650ce459dbb923601896143d1/users/01f362a125834841b628bf6dd3d1d5b2/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.683445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b3c9f-ac0d-4d87-af6f-22d39fedf910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41193db650ce459dbb923601896143d1, user_id=01f362a125834841b628bf6dd3d1d5b2, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.686316 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b3c9f-ac0d-4d87-af6f-22d39fedf910 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.705063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1285/2573] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:24 2022] PUT /identity/v3/projects/41193db650ce459dbb923601896143d1/users/01f362a125834841b628bf6dd3d1d5b2/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:24.710126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1916-5026-4a08-b2f9-8e81347498b6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.717612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:24.721946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1916-5026-4a08-b2f9-8e81347498b6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.796170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f2ec-2dbc-4a1e-8c95-28c0c25319e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.796907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f2ec-2dbc-4a1e-8c95-28c0c25319e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.797183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f2ec-2dbc-4a1e-8c95-28c0c25319e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.797450 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f2ec-2dbc-4a1e-8c95-28c0c25319e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.798088 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536f2ec-2dbc-4a1e-8c95-28c0c25319e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.799699 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536f2ec-2dbc-4a1e-8c95-28c0c25319e6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.812348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1289/2574] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:24.816209 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b3c9f-ac0d-4d87-af6f-22d39fedf910 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.829355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b3c9f-ac0d-4d87-af6f-22d39fedf910 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:24.834721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4196e30-3e91-4136-86e8-5c4c85478086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:24.835399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4196e30-3e91-4136-86e8-5c4c85478086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.835678 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4196e30-3e91-4136-86e8-5c4c85478086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.836001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4196e30-3e91-4136-86e8-5c4c85478086 admin admin] PATH_INFO: `/v3/projects/41193db650ce459dbb923601896143d1/users/01f362a125834841b628bf6dd3d1d5b2/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.838118 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4196e30-3e91-4136-86e8-5c4c85478086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41193db650ce459dbb923601896143d1, user_id=01f362a125834841b628bf6dd3d1d5b2, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:24.840852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4196e30-3e91-4136-86e8-5c4c85478086 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:24.851960 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1286/2575] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:24 2022] PUT /identity/v3/projects/41193db650ce459dbb923601896143d1/users/01f362a125834841b628bf6dd3d1d5b2/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:24.864673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f580fb-ca4d-488e-bfc2-17e1d6bb6e9e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:24.865030 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f580fb-ca4d-488e-bfc2-17e1d6bb6e9e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:24.868258 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f580fb-ca4d-488e-bfc2-17e1d6bb6e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:24.896225 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-83f580fb-ca4d-488e-bfc2-17e1d6bb6e9e None None] MFA Rules not processed for user `01f362a125834841b628bf6dd3d1d5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:24.897569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-83f580fb-ca4d-488e-bfc2-17e1d6bb6e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:24.968023 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1290/2576] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:24.980011 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4196e30-3e91-4136-86e8-5c4c85478086 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:24.981816 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b4196e30-3e91-4136-86e8-5c4c85478086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:25.042036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4196e30-3e91-4136-86e8-5c4c85478086 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:25.045796 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba847c80-5803-4747-bf9b-608d5ed30690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:25.046393 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba847c80-5803-4747-bf9b-608d5ed30690 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:25.046713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba847c80-5803-4747-bf9b-608d5ed30690 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:25.046984 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba847c80-5803-4747-bf9b-608d5ed30690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:25.056762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba847c80-5803-4747-bf9b-608d5ed30690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:25.058329 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba847c80-5803-4747-bf9b-608d5ed30690 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:25.068291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1287/2577] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:25.148673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f580fb-ca4d-488e-bfc2-17e1d6bb6e9e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:25.158433 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f580fb-ca4d-488e-bfc2-17e1d6bb6e9e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:25.162202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fc47a-976a-4aee-a018-f9e1898388a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:25.162798 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fc47a-976a-4aee-a018-f9e1898388a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:25.163086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fc47a-976a-4aee-a018-f9e1898388a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:25.163362 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fc47a-976a-4aee-a018-f9e1898388a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:25.222185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fc47a-976a-4aee-a018-f9e1898388a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:25.223712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fc47a-976a-4aee-a018-f9e1898388a6 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:25.233510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1291/2578] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:27.570925 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ed1b5-40a5-4ecf-98fb-c6d44a9abc55 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:27.571410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ed1b5-40a5-4ecf-98fb-c6d44a9abc55 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:27.571519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ed1b5-40a5-4ecf-98fb-c6d44a9abc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:27.592865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-697ed1b5-40a5-4ecf-98fb-c6d44a9abc55 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:27.594131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-697ed1b5-40a5-4ecf-98fb-c6d44a9abc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:27.599094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:27.656636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1288/2579] 10.222.0.75 () {58 vars in 967 bytes} [Fri Aug 19 15:30:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:27.702436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fc47a-976a-4aee-a018-f9e1898388a6 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:27.703944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a65fc47a-976a-4aee-a018-f9e1898388a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:27.734715 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:27.777375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fc47a-976a-4aee-a018-f9e1898388a6 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:27.782380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1025e9-a730-425b-9df6-79b235ba244e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:27.782978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1025e9-a730-425b-9df6-79b235ba244e service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:27.783330 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1025e9-a730-425b-9df6-79b235ba244e service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:27.783599 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1025e9-a730-425b-9df6-79b235ba244e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:27.792463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1025e9-a730-425b-9df6-79b235ba244e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:27.794069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1025e9-a730-425b-9df6-79b235ba244e service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:27.804464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1292/2580] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:30:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:30:27.983639 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aac68a-daf4-4727-a0d3-79359b96d7e2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:27.984132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aac68a-daf4-4727-a0d3-79359b96d7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:27.984429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aac68a-daf4-4727-a0d3-79359b96d7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:28.007842 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-83aac68a-daf4-4727-a0d3-79359b96d7e2 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:28.009095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-83aac68a-daf4-4727-a0d3-79359b96d7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:28.017648 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1289/2581] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:30:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:28.030410 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1025e9-a730-425b-9df6-79b235ba244e service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:28.043708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1025e9-a730-425b-9df6-79b235ba244e service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:28.050582 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a5158-7133-45c1-9d45-3cc186f24bdb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:28.051973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a5158-7133-45c1-9d45-3cc186f24bdb service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:28.052252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a5158-7133-45c1-9d45-3cc186f24bdb service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:28.052746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a5158-7133-45c1-9d45-3cc186f24bdb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:28.063525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a5158-7133-45c1-9d45-3cc186f24bdb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:28.065311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a5158-7133-45c1-9d45-3cc186f24bdb service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:28.075032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1293/2582] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:30:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:30:29.592563 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aac68a-daf4-4727-a0d3-79359b96d7e2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:29.594609 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-83aac68a-daf4-4727-a0d3-79359b96d7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:29.765803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aac68a-daf4-4727-a0d3-79359b96d7e2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:29.775099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:29.776117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:29.776694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:29.777172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] PATH_INFO: `/v3/users/86c0d10820ed45b9960704a6c4de2b6d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:29.778576 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c0d10820ed45b9960704a6c4de2b6d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:29.781191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:29.875432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c0d10820ed45b9960704a6c4de2b6d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:29.890540 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c0d10820ed45b9960704a6c4de2b6d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:29.894112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c0d10820ed45b9960704a6c4de2b6d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:29.923243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c0d10820ed45b9960704a6c4de2b6d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:29.949781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c0d10820ed45b9960704a6c4de2b6d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:29.971115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1290/2583] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:29 2022] DELETE /identity/v3/users/86c0d10820ed45b9960704a6c4de2b6d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:29.974665 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c019f-d04f-4ddd-900f-c5864bf3b0ca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:29.975185 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c019f-d04f-4ddd-900f-c5864bf3b0ca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:29.975555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c019f-d04f-4ddd-900f-c5864bf3b0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:30.031691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b87c019f-d04f-4ddd-900f-c5864bf3b0ca None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:30.033049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b87c019f-d04f-4ddd-900f-c5864bf3b0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:30.185115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1294/2584] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:30.196125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:30.198081 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:30.260104 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:30.341012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541c1ac-b097-4f9c-beb8-05ae47b6ca04 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:30.353479 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd2319-4c9a-49e5-8444-f9457f775077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:30.354282 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2319-4c9a-49e5-8444-f9457f775077 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:30.354571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2319-4c9a-49e5-8444-f9457f775077 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:30.354908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd2319-4c9a-49e5-8444-f9457f775077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:30.379175 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd2319-4c9a-49e5-8444-f9457f775077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:30.380887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd2319-4c9a-49e5-8444-f9457f775077 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:30.402467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1291/2585] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:30.994052 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c019f-d04f-4ddd-900f-c5864bf3b0ca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:30.996996 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b87c019f-d04f-4ddd-900f-c5864bf3b0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:31.037347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c019f-d04f-4ddd-900f-c5864bf3b0ca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:31.066216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:31.070286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:31.070567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:31.070847 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] PATH_INFO: `/v3/projects/05932e06378e4003971ac1b659107747` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:31.073093 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05932e06378e4003971ac1b659107747)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:31.075311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:31.188485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05932e06378e4003971ac1b659107747', 'request_id': 'req-99c6a8ec-f905-4742-a622-666cfd1159c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:31.200567 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05932e06378e4003971ac1b659107747', 'request_id': 'req-99c6a8ec-f905-4742-a622-666cfd1159c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:31.219626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05932e06378e4003971ac1b659107747', 'request_id': 'req-99c6a8ec-f905-4742-a622-666cfd1159c2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:31.261139 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] The token cache is being invalidate because project 05932e06378e4003971ac1b659107747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:31.262634 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:31.264547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-99c6a8ec-f905-4742-a622-666cfd1159c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:31.267374 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1295/2586] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:30 2022] DELETE /identity/v3/projects/05932e06378e4003971ac1b659107747 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:31.273543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c4c71-7546-4741-9d1f-7dd88fa83bf6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:31.274057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c4c71-7546-4741-9d1f-7dd88fa83bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:31.274156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c4c71-7546-4741-9d1f-7dd88fa83bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:31.328872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2c8c4c71-7546-4741-9d1f-7dd88fa83bf6 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:31.330792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2c8c4c71-7546-4741-9d1f-7dd88fa83bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:31.444423 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1292/2587] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:31.448970 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb702-da31-478a-a6dd-e8610f5374d8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:31.449260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb702-da31-478a-a6dd-e8610f5374d8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:31.449537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb702-da31-478a-a6dd-e8610f5374d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:31.505455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8a5eb702-da31-478a-a6dd-e8610f5374d8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:31.506719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a5eb702-da31-478a-a6dd-e8610f5374d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:31.528148 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1296/2588] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:31.533289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c4c71-7546-4741-9d1f-7dd88fa83bf6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:31.554040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c4c71-7546-4741-9d1f-7dd88fa83bf6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:31.561580 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda54e9-6e1d-4296-a51f-6e1bece0d60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:31.564082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda54e9-6e1d-4296-a51f-6e1bece0d60e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:31.564584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda54e9-6e1d-4296-a51f-6e1bece0d60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:31.565061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda54e9-6e1d-4296-a51f-6e1bece0d60e admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:31.565903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda54e9-6e1d-4296-a51f-6e1bece0d60e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:30:31.567309 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda54e9-6e1d-4296-a51f-6e1bece0d60e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:31.569333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda54e9-6e1d-4296-a51f-6e1bece0d60e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:31.592600 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1293/2589] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:30:31.597555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5eb702-da31-478a-a6dd-e8610f5374d8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:31.627693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5eb702-da31-478a-a6dd-e8610f5374d8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:31.652465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0546149-3100-489e-89f8-5fe7ed665c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:31.655039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0546149-3100-489e-89f8-5fe7ed665c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:31.655177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0546149-3100-489e-89f8-5fe7ed665c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:31.655361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0546149-3100-489e-89f8-5fe7ed665c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:31.658780 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0546149-3100-489e-89f8-5fe7ed665c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:31.662817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0546149-3100-489e-89f8-5fe7ed665c2c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:31.700925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:31.727453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1297/2590] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:31 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:31.738059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda54e9-6e1d-4296-a51f-6e1bece0d60e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:31.757122 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda54e9-6e1d-4296-a51f-6e1bece0d60e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:31.862780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce206869-5994-422b-adce-40f4517318aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:31.864592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206869-5994-422b-adce-40f4517318aa admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:31.865057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206869-5994-422b-adce-40f4517318aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:31.865708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206869-5994-422b-adce-40f4517318aa admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:31.867061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce206869-5994-422b-adce-40f4517318aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:31.869298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce206869-5994-422b-adce-40f4517318aa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:31.906888 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1294/2591] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:31 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:31.910564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0546149-3100-489e-89f8-5fe7ed665c2c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:31.926876 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0546149-3100-489e-89f8-5fe7ed665c2c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:31.938353 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1153-b61b-4e1f-9515-6f9288158b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:31.941102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1153-b61b-4e1f-9515-6f9288158b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:31.942766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1153-b61b-4e1f-9515-6f9288158b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:31.943254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1153-b61b-4e1f-9515-6f9288158b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:31.944355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d1153-b61b-4e1f-9515-6f9288158b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:31.947089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d1153-b61b-4e1f-9515-6f9288158b85 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:31.963377 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1298/2592] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:31.967111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce206869-5994-422b-adce-40f4517318aa admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:31.993538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce206869-5994-422b-adce-40f4517318aa admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:32.012543 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50690b40-0b61-4832-bc50-337f68d7160e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:32.013447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50690b40-0b61-4832-bc50-337f68d7160e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:32.014692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50690b40-0b61-4832-bc50-337f68d7160e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:32.015967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50690b40-0b61-4832-bc50-337f68d7160e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:32.016835 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50690b40-0b61-4832-bc50-337f68d7160e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:32.018789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50690b40-0b61-4832-bc50-337f68d7160e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:32.039146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1295/2593] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:32.043334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1153-b61b-4e1f-9515-6f9288158b85 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:32.060742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1153-b61b-4e1f-9515-6f9288158b85 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:32.074806 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd601e-ba42-4f27-89cf-62d1c0f36793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:32.075480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd601e-ba42-4f27-89cf-62d1c0f36793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:32.075759 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd601e-ba42-4f27-89cf-62d1c0f36793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:32.076046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd601e-ba42-4f27-89cf-62d1c0f36793 admin admin] PATH_INFO: `/v3/projects/d2ba9fce1d2948d3b7ab9cea5d936ea4/users/d348fd6ae41b4112b8dad30a907738b3/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:32.130264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd601e-ba42-4f27-89cf-62d1c0f36793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ba9fce1d2948d3b7ab9cea5d936ea4, user_id=d348fd6ae41b4112b8dad30a907738b3, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:32.136407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd601e-ba42-4f27-89cf-62d1c0f36793 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:32.181420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1299/2594] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:32 2022] PUT /identity/v3/projects/d2ba9fce1d2948d3b7ab9cea5d936ea4/users/d348fd6ae41b4112b8dad30a907738b3/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:32.183599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd20b3-6672-4b09-a208-1d116f4eee0d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:32.183887 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd20b3-6672-4b09-a208-1d116f4eee0d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:32.184164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd20b3-6672-4b09-a208-1d116f4eee0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:32.247734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-72fd20b3-6672-4b09-a208-1d116f4eee0d None None] MFA Rules not processed for user `d348fd6ae41b4112b8dad30a907738b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:32.249048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-72fd20b3-6672-4b09-a208-1d116f4eee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:32.337604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:32.477546 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1296/2595] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:32.489952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd601e-ba42-4f27-89cf-62d1c0f36793 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:32.492682 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8ebd601e-ba42-4f27-89cf-62d1c0f36793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:32.673236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd601e-ba42-4f27-89cf-62d1c0f36793 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:32.688166 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c7120-e30a-41cb-bed8-8e7d8c327d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:32.689499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c7120-e30a-41cb-bed8-8e7d8c327d79 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:32.689896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c7120-e30a-41cb-bed8-8e7d8c327d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:32.690277 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c7120-e30a-41cb-bed8-8e7d8c327d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:32.708881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c7120-e30a-41cb-bed8-8e7d8c327d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:32.715214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c7120-e30a-41cb-bed8-8e7d8c327d79 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:32.727639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1300/2596] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:36.275910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd20b3-6672-4b09-a208-1d116f4eee0d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:36.278168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-72fd20b3-6672-4b09-a208-1d116f4eee0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:36.434916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd20b3-6672-4b09-a208-1d116f4eee0d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:36.441187 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:36.442709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:36.443205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:36.443558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] PATH_INFO: `/v3/users/c6edfcff5efc4434ba0c6f50271cd7c5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:36.444859 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6edfcff5efc4434ba0c6f50271cd7c5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:36.449589 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:36.552035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6edfcff5efc4434ba0c6f50271cd7c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:36.563905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6edfcff5efc4434ba0c6f50271cd7c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:36.565172 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6edfcff5efc4434ba0c6f50271cd7c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:36.593608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6edfcff5efc4434ba0c6f50271cd7c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:36.620036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6edfcff5efc4434ba0c6f50271cd7c5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:36.633883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1297/2597] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:36 2022] DELETE /identity/v3/users/c6edfcff5efc4434ba0c6f50271cd7c5 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:36.638242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77fc42-eaf8-4768-9132-5d747b442872 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:36.638749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77fc42-eaf8-4768-9132-5d747b442872 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:36.638847 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77fc42-eaf8-4768-9132-5d747b442872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:36.673132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bf77fc42-eaf8-4768-9132-5d747b442872 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:36.673132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf77fc42-eaf8-4768-9132-5d747b442872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:36.811110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1301/2598] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:36.822869 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:36.824439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:36.963969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:36.992871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb2a5c-cb88-4cf4-8baf-01456b879d83 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:37.001217 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eecf2d-03f0-41f3-8c93-669262a5f01d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:37.002043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eecf2d-03f0-41f3-8c93-669262a5f01d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:37.002542 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eecf2d-03f0-41f3-8c93-669262a5f01d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:37.002973 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eecf2d-03f0-41f3-8c93-669262a5f01d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:37.022733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eecf2d-03f0-41f3-8c93-669262a5f01d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:37.025845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eecf2d-03f0-41f3-8c93-669262a5f01d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:37.046878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1298/2599] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:38.041067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77fc42-eaf8-4768-9132-5d747b442872 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:38.042557 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf77fc42-eaf8-4768-9132-5d747b442872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:38.064044 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77fc42-eaf8-4768-9132-5d747b442872 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:38.076779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:38.077892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:38.078369 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:38.078851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] PATH_INFO: `/v3/projects/22f6cef1460e465fa102cecbd3904283` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:38.080223 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f6cef1460e465fa102cecbd3904283)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:38.089903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:38.244710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f6cef1460e465fa102cecbd3904283', 'request_id': 'req-4a605c27-f873-4740-8297-f0ce42041478'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.262881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f6cef1460e465fa102cecbd3904283', 'request_id': 'req-4a605c27-f873-4740-8297-f0ce42041478'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.266211 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:38.286074 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f6cef1460e465fa102cecbd3904283', 'request_id': 'req-4a605c27-f873-4740-8297-f0ce42041478'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.306282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] The token cache is being invalidate because project 22f6cef1460e465fa102cecbd3904283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:38.307159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.308779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.311785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1302/2600] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:38 2022] DELETE /identity/v3/projects/22f6cef1460e465fa102cecbd3904283 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:38.317820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eecf2d-03f0-41f3-8c93-669262a5f01d service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:38.330133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eecf2d-03f0-41f3-8c93-669262a5f01d service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:38.420287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:38.421026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:38.421599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:38.421882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] PATH_INFO: `/v3/users/01f362a125834841b628bf6dd3d1d5b2` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:38.423311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f362a125834841b628bf6dd3d1d5b2)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:38.425598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:38.592961 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f362a125834841b628bf6dd3d1d5b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.633729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f362a125834841b628bf6dd3d1d5b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.636100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f362a125834841b628bf6dd3d1d5b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.691198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f362a125834841b628bf6dd3d1d5b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.704295 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-75e44203-0fdd-429f-9899-e04a89a7ce82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f362a125834841b628bf6dd3d1d5b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:38.730120 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1299/2601] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:38 2022] DELETE /identity/v3/users/01f362a125834841b628bf6dd3d1d5b2 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:39.356675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:39.358920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:39.461393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a605c27-f873-4740-8297-f0ce42041478 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:39.713494 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:39.714689 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:39.715082 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:39.715459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] PATH_INFO: `/v3/projects/41193db650ce459dbb923601896143d1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:39.717717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=41193db650ce459dbb923601896143d1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:39.722735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:39.857673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41193db650ce459dbb923601896143d1', 'request_id': 'req-1fe95032-907d-4120-968b-05528a9ab69d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:39.869057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41193db650ce459dbb923601896143d1', 'request_id': 'req-1fe95032-907d-4120-968b-05528a9ab69d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:39.882857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41193db650ce459dbb923601896143d1', 'request_id': 'req-1fe95032-907d-4120-968b-05528a9ab69d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:39.922460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] The token cache is being invalidate because project 41193db650ce459dbb923601896143d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:39.922997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:39.924588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1fe95032-907d-4120-968b-05528a9ab69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:39.941697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1303/2602] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:39 2022] DELETE /identity/v3/projects/41193db650ce459dbb923601896143d1 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:39.945219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949fa60-c44d-4395-a0c6-023384da4759 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:39.945751 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949fa60-c44d-4395-a0c6-023384da4759 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:39.947668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949fa60-c44d-4395-a0c6-023384da4759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:40.007954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5949fa60-c44d-4395-a0c6-023384da4759 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:40.016411 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5949fa60-c44d-4395-a0c6-023384da4759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:40.270190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1300/2603] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:40.272308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba85d5-ebdb-4f4c-be89-80b9e38b3c6d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:40.272704 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba85d5-ebdb-4f4c-be89-80b9e38b3c6d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:40.272911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba85d5-ebdb-4f4c-be89-80b9e38b3c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:40.310949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-22ba85d5-ebdb-4f4c-be89-80b9e38b3c6d None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:40.312136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22ba85d5-ebdb-4f4c-be89-80b9e38b3c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:40.337234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1304/2604] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:40.341502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949fa60-c44d-4395-a0c6-023384da4759 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:40.354392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949fa60-c44d-4395-a0c6-023384da4759 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:40.369829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d395b-b516-42f8-a330-9335acecdf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:40.370470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d395b-b516-42f8-a330-9335acecdf63 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:40.370822 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d395b-b516-42f8-a330-9335acecdf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:40.371203 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d395b-b516-42f8-a330-9335acecdf63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:40.371852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d395b-b516-42f8-a330-9335acecdf63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:30:40.372149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d395b-b516-42f8-a330-9335acecdf63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:40.374046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d395b-b516-42f8-a330-9335acecdf63 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:40.375920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:40.422607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1301/2605] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:30:40.424808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba85d5-ebdb-4f4c-be89-80b9e38b3c6d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:40.447461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba85d5-ebdb-4f4c-be89-80b9e38b3c6d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:40.480263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3077a8-51a2-4bbc-a8ab-9c80e0600aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:40.481334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3077a8-51a2-4bbc-a8ab-9c80e0600aff admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:40.481648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3077a8-51a2-4bbc-a8ab-9c80e0600aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:40.481978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3077a8-51a2-4bbc-a8ab-9c80e0600aff admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:40.482638 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3077a8-51a2-4bbc-a8ab-9c80e0600aff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:40.487732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3077a8-51a2-4bbc-a8ab-9c80e0600aff admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:40.554830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1305/2606] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:40 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:40.558691 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d395b-b516-42f8-a330-9335acecdf63 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:40.575088 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d395b-b516-42f8-a330-9335acecdf63 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:40.705406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d867f1-31e8-4e14-9f71-ca3ab95e3104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:40.706483 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d867f1-31e8-4e14-9f71-ca3ab95e3104 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:40.706995 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d867f1-31e8-4e14-9f71-ca3ab95e3104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:40.708168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d867f1-31e8-4e14-9f71-ca3ab95e3104 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:40.709132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d867f1-31e8-4e14-9f71-ca3ab95e3104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:40.712137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d867f1-31e8-4e14-9f71-ca3ab95e3104 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:40.755284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1302/2607] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:40 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:40.759129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3077a8-51a2-4bbc-a8ab-9c80e0600aff admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:40.782440 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3077a8-51a2-4bbc-a8ab-9c80e0600aff admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:40.798221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0545f-5d41-46ef-8b20-ba04fd878f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:40.812733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0545f-5d41-46ef-8b20-ba04fd878f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:40.813609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0545f-5d41-46ef-8b20-ba04fd878f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:40.814221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0545f-5d41-46ef-8b20-ba04fd878f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:40.815417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0545f-5d41-46ef-8b20-ba04fd878f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:40.833821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0545f-5d41-46ef-8b20-ba04fd878f91 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:40.862121 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1306/2608] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:40.869504 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d867f1-31e8-4e14-9f71-ca3ab95e3104 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:40.882716 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d867f1-31e8-4e14-9f71-ca3ab95e3104 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:40.891651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1000b-45f3-4c93-aaec-f2569bd3bdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:40.892256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1000b-45f3-4c93-aaec-f2569bd3bdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:40.892568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1000b-45f3-4c93-aaec-f2569bd3bdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:40.892839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1000b-45f3-4c93-aaec-f2569bd3bdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:40.893485 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1000b-45f3-4c93-aaec-f2569bd3bdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:40.895673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1000b-45f3-4c93-aaec-f2569bd3bdb4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:40.907946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1303/2609] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:40.913496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0545f-5d41-46ef-8b20-ba04fd878f91 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:40.929695 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0545f-5d41-46ef-8b20-ba04fd878f91 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:40.934333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092208f-2751-4680-8724-212967623fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:40.934990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092208f-2751-4680-8724-212967623fba admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:40.935264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092208f-2751-4680-8724-212967623fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:40.935541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092208f-2751-4680-8724-212967623fba admin admin] PATH_INFO: `/v3/projects/b69a23d69346439982e859fe1db60341/users/87b410b34d134b7f90cf658292e4bf1f/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:40.963180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092208f-2751-4680-8724-212967623fba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69a23d69346439982e859fe1db60341, user_id=87b410b34d134b7f90cf658292e4bf1f, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:40.975478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1000b-45f3-4c93-aaec-f2569bd3bdb4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:40.977840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092208f-2751-4680-8724-212967623fba admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:40.978114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-27c1000b-45f3-4c93-aaec-f2569bd3bdb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:41.007494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1000b-45f3-4c93-aaec-f2569bd3bdb4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:41.025484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1307/2610] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:40 2022] PUT /identity/v3/projects/b69a23d69346439982e859fe1db60341/users/87b410b34d134b7f90cf658292e4bf1f/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:30:41.035593 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45123e54-0d0c-41dd-be09-e487de14a997 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:41.035922 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45123e54-0d0c-41dd-be09-e487de14a997 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:41.036204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45123e54-0d0c-41dd-be09-e487de14a997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:41.126562 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:41.138227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-45123e54-0d0c-41dd-be09-e487de14a997 None None] MFA Rules not processed for user `87b410b34d134b7f90cf658292e4bf1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:41.146903 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-45123e54-0d0c-41dd-be09-e487de14a997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:41.158530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:41.159159 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:41.159443 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:41.159719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] PATH_INFO: `/v3/users/48b7030c933c4e6a92711b35cd6f9685` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:41.165208 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b7030c933c4e6a92711b35cd6f9685)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:41.167405 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:41.316987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b7030c933c4e6a92711b35cd6f9685'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:41.353276 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b7030c933c4e6a92711b35cd6f9685'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:41.355620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b7030c933c4e6a92711b35cd6f9685'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:41.406125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b7030c933c4e6a92711b35cd6f9685'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:41.406956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1308/2611] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:41.407892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:41.416273 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45123e54-0d0c-41dd-be09-e487de14a997 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:41.429146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b7030c933c4e6a92711b35cd6f9685'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:41.447307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45123e54-0d0c-41dd-be09-e487de14a997 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:41.453953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1304/2612] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:40 2022] DELETE /identity/v3/users/48b7030c933c4e6a92711b35cd6f9685 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:41.519427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4b4aa-bb69-4fbb-bde1-a625ac2112b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:41.522264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4b4aa-bb69-4fbb-bde1-a625ac2112b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:41.525412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4b4aa-bb69-4fbb-bde1-a625ac2112b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:41.526968 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4b4aa-bb69-4fbb-bde1-a625ac2112b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:41.527873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4b4aa-bb69-4fbb-bde1-a625ac2112b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:41.535692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4b4aa-bb69-4fbb-bde1-a625ac2112b5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:41.703065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1309/2613] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:41 2022] POST /identity/v3/projects => generated 387 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:41.704169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:41.706152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:41.856257 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0cad6-fa30-4c0b-ac7b-267e742f1f27 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:41.896092 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4b4aa-bb69-4fbb-bde1-a625ac2112b5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:41.898043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-51b4b4aa-bb69-4fbb-bde1-a625ac2112b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:41.993292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec950d-8acf-4d61-80fb-babe4d9e4443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:41.993714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec950d-8acf-4d61-80fb-babe4d9e4443 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:41.993714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec950d-8acf-4d61-80fb-babe4d9e4443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:41.994141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec950d-8acf-4d61-80fb-babe4d9e4443 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:41.995283 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec950d-8acf-4d61-80fb-babe4d9e4443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:41.997885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec950d-8acf-4d61-80fb-babe4d9e4443 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:42.039066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1305/2614] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:41 2022] POST /identity/v3/users => generated 300 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:42.042823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec950d-8acf-4d61-80fb-babe4d9e4443 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:42.055993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec950d-8acf-4d61-80fb-babe4d9e4443 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:42.063700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92037e-5147-4e2d-8d8d-a354e26bc590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:42.069480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92037e-5147-4e2d-8d8d-a354e26bc590 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.070287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92037e-5147-4e2d-8d8d-a354e26bc590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.071087 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92037e-5147-4e2d-8d8d-a354e26bc590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.072166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92037e-5147-4e2d-8d8d-a354e26bc590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:42.073896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92037e-5147-4e2d-8d8d-a354e26bc590 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:42.078094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4b4aa-bb69-4fbb-bde1-a625ac2112b5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:42.088661 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1306/2615] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:42.092089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:42.092814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.093199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.093735 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] PATH_INFO: `/v3/projects/3c3e603ab69848a2b9688f1e536017f3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.094655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92037e-5147-4e2d-8d8d-a354e26bc590 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:42.095478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c3e603ab69848a2b9688f1e536017f3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:42.099838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:42.121834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92037e-5147-4e2d-8d8d-a354e26bc590 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:42.135340 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa66f64-6dfa-4939-bab7-e66435740de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:42.136840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa66f64-6dfa-4939-bab7-e66435740de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.137318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa66f64-6dfa-4939-bab7-e66435740de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.137874 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa66f64-6dfa-4939-bab7-e66435740de6 admin admin] PATH_INFO: `/v3/projects/2de80cacfcf14d6f946daa8b57826f30/users/d0d04f221c224244adf8029001d669f2/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.162330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa66f64-6dfa-4939-bab7-e66435740de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de80cacfcf14d6f946daa8b57826f30, user_id=d0d04f221c224244adf8029001d669f2, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:42.165476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa66f64-6dfa-4939-bab7-e66435740de6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:42.184663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1307/2616] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:42 2022] PUT /identity/v3/projects/2de80cacfcf14d6f946daa8b57826f30/users/d0d04f221c224244adf8029001d669f2/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:42.185682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa66f64-6dfa-4939-bab7-e66435740de6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:42.196486 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c3e603ab69848a2b9688f1e536017f3', 'request_id': 'req-80d82e94-5c6f-4546-a24a-905383144374'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.212237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c3e603ab69848a2b9688f1e536017f3', 'request_id': 'req-80d82e94-5c6f-4546-a24a-905383144374'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.225181 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c3e603ab69848a2b9688f1e536017f3', 'request_id': 'req-80d82e94-5c6f-4546-a24a-905383144374'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.243319 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] The token cache is being invalidate because project 3c3e603ab69848a2b9688f1e536017f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:42.243652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.245551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.250918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1310/2617] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:41 2022] DELETE /identity/v3/projects/3c3e603ab69848a2b9688f1e536017f3 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:30:42.251981 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:42.255804 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:42.281194 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:42.290582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:42.305074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa66f64-6dfa-4939-bab7-e66435740de6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:42.313765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc3a5b-5d4d-4672-b484-4a57650ada77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:42.315473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc3a5b-5d4d-4672-b484-4a57650ada77 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.315860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc3a5b-5d4d-4672-b484-4a57650ada77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.316305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc3a5b-5d4d-4672-b484-4a57650ada77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.317873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc3a5b-5d4d-4672-b484-4a57650ada77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:42.320165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc3a5b-5d4d-4672-b484-4a57650ada77 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:42.336624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1308/2618] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:42.339645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc3a5b-5d4d-4672-b484-4a57650ada77 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:42.341707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c8cc3a5b-5d4d-4672-b484-4a57650ada77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:42.355391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc3a5b-5d4d-4672-b484-4a57650ada77 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:42.362771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bc702-fbf8-4b1b-bb96-132a9994cb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:42.364041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc702-fbf8-4b1b-bb96-132a9994cb1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.364529 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc702-fbf8-4b1b-bb96-132a9994cb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.364813 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc702-fbf8-4b1b-bb96-132a9994cb1c admin admin] PATH_INFO: `/v3/projects/2de80cacfcf14d6f946daa8b57826f30/users/d0d04f221c224244adf8029001d669f2/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.366789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bc702-fbf8-4b1b-bb96-132a9994cb1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de80cacfcf14d6f946daa8b57826f30, user_id=d0d04f221c224244adf8029001d669f2, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:42.368295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d82e94-5c6f-4546-a24a-905383144374 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:42.369519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bc702-fbf8-4b1b-bb96-132a9994cb1c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:42.378460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:42.380418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.380713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.380993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] PATH_INFO: `/v3/users/7e5eb15ec8514b4b97cee01f9d360cfe` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.382249 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5eb15ec8514b4b97cee01f9d360cfe)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:42.383231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1309/2619] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:42 2022] PUT /identity/v3/projects/2de80cacfcf14d6f946daa8b57826f30/users/d0d04f221c224244adf8029001d669f2/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:30:42.385881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:42.394212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a03911-69bf-4dc2-883d-826f7ad8faf8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.394493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a03911-69bf-4dc2-883d-826f7ad8faf8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.394931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a03911-69bf-4dc2-883d-826f7ad8faf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.427621 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c0a03911-69bf-4dc2-883d-826f7ad8faf8 None None] MFA Rules not processed for user `d0d04f221c224244adf8029001d669f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:42.428945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c0a03911-69bf-4dc2-883d-826f7ad8faf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:42.456204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5eb15ec8514b4b97cee01f9d360cfe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.457149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5eb15ec8514b4b97cee01f9d360cfe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.458351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5eb15ec8514b4b97cee01f9d360cfe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.468066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5eb15ec8514b4b97cee01f9d360cfe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.492846 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5eb15ec8514b4b97cee01f9d360cfe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:42.511742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1311/2620] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:42 2022] DELETE /identity/v3/users/7e5eb15ec8514b4b97cee01f9d360cfe => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:42.544578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1310/2621] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:42.558651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:42.560200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:42.645945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544bc979-d9f7-4ff2-a64e-43c863c61bf2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:42.651396 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aec26-ff22-42cf-ba5b-37ea646f0ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:42.652740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aec26-ff22-42cf-ba5b-37ea646f0ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.653024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aec26-ff22-42cf-ba5b-37ea646f0ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.653303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aec26-ff22-42cf-ba5b-37ea646f0ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.731393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:42.747544 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a03911-69bf-4dc2-883d-826f7ad8faf8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:42.748836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c0a03911-69bf-4dc2-883d-826f7ad8faf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:42.760194 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6aec26-ff22-42cf-ba5b-37ea646f0ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:42.762298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6aec26-ff22-42cf-ba5b-37ea646f0ea8 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:42.784767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1312/2622] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:42.864323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aec26-ff22-42cf-ba5b-37ea646f0ea8 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:42.866122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-df6aec26-ff22-42cf-ba5b-37ea646f0ea8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:42.882638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a03911-69bf-4dc2-883d-826f7ad8faf8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:42.977141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aec26-ff22-42cf-ba5b-37ea646f0ea8 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:42.982888 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:42.983311 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:42.984152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.984696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.985298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] PATH_INFO: `/v3/users/87b410b34d134b7f90cf658292e4bf1f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.986854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b410b34d134b7f90cf658292e4bf1f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:42.987166 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:42.988501 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:42.988867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] PATH_INFO: `/v3/projects/036bcceabe0f4077a91d5c5d4bdd1b11` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:42.990273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=036bcceabe0f4077a91d5c5d4bdd1b11)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:42.990726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:42.997601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:43.011965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:43.085032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b410b34d134b7f90cf658292e4bf1f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.085734 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b410b34d134b7f90cf658292e4bf1f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.086667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b410b34d134b7f90cf658292e4bf1f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.090944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036bcceabe0f4077a91d5c5d4bdd1b11', 'request_id': 'req-c560bc62-1e6f-49ac-a067-0e5881f092cb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.102004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b410b34d134b7f90cf658292e4bf1f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.103963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036bcceabe0f4077a91d5c5d4bdd1b11', 'request_id': 'req-c560bc62-1e6f-49ac-a067-0e5881f092cb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.126060 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036bcceabe0f4077a91d5c5d4bdd1b11', 'request_id': 'req-c560bc62-1e6f-49ac-a067-0e5881f092cb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.129977 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b410b34d134b7f90cf658292e4bf1f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.149184 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] The token cache is being invalidate because project 036bcceabe0f4077a91d5c5d4bdd1b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:43.149732 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.150809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c560bc62-1e6f-49ac-a067-0e5881f092cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.153269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1311/2623] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:42 2022] DELETE /identity/v3/projects/036bcceabe0f4077a91d5c5d4bdd1b11 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:43.204848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1313/2624] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:42 2022] DELETE /identity/v3/users/87b410b34d134b7f90cf658292e4bf1f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:43.206795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd4f73-9356-48eb-8654-f90ccb092a1c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:43.207857 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd4f73-9356-48eb-8654-f90ccb092a1c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:43.208298 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd4f73-9356-48eb-8654-f90ccb092a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:43.244534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a2fd4f73-9356-48eb-8654-f90ccb092a1c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:43.245740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a2fd4f73-9356-48eb-8654-f90ccb092a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:43.339144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1312/2625] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:43.355496 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:43.356882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:43.477879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d736ac-300f-4793-ba4f-335a11e250f8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:43.481600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81e344-d0ce-495a-aae8-81e67823ba82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:43.482208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81e344-d0ce-495a-aae8-81e67823ba82 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:43.482485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81e344-d0ce-495a-aae8-81e67823ba82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:43.482787 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81e344-d0ce-495a-aae8-81e67823ba82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:43.493036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81e344-d0ce-495a-aae8-81e67823ba82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:43.494485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81e344-d0ce-495a-aae8-81e67823ba82 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:43.505276 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1314/2626] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:43.835579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd4f73-9356-48eb-8654-f90ccb092a1c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:43.837287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a2fd4f73-9356-48eb-8654-f90ccb092a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:43.869687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd4f73-9356-48eb-8654-f90ccb092a1c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:43.876917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:43.877635 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:43.878014 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:43.878359 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] PATH_INFO: `/v3/projects/b69a23d69346439982e859fe1db60341` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:43.879607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69a23d69346439982e859fe1db60341)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:43.881709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:43.951463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69a23d69346439982e859fe1db60341', 'request_id': 'req-ae353cd4-bf61-4c7d-801b-02fd68de57cd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.957992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69a23d69346439982e859fe1db60341', 'request_id': 'req-ae353cd4-bf61-4c7d-801b-02fd68de57cd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.968116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69a23d69346439982e859fe1db60341', 'request_id': 'req-ae353cd4-bf61-4c7d-801b-02fd68de57cd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.982894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] The token cache is being invalidate because project b69a23d69346439982e859fe1db60341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:43.982894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.982894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:43.984117 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1313/2627] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:43 2022] DELETE /identity/v3/projects/b69a23d69346439982e859fe1db60341 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:43.988202 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81e344-d0ce-495a-aae8-81e67823ba82 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:43.994241 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:44.002270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81e344-d0ce-495a-aae8-81e67823ba82 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:44.073681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:44.074324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:44.074685 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:44.075035 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] PATH_INFO: `/v3/users/d0d04f221c224244adf8029001d669f2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:44.077077 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d04f221c224244adf8029001d669f2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:44.079885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:44.132832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d04f221c224244adf8029001d669f2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.133654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d04f221c224244adf8029001d669f2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.134334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d04f221c224244adf8029001d669f2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.142126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d04f221c224244adf8029001d669f2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.164571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0f571388-3d95-4c99-9e7a-b490cc932f32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d04f221c224244adf8029001d669f2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.187097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1315/2628] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:43 2022] DELETE /identity/v3/users/d0d04f221c224244adf8029001d669f2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:30:44.483907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:44.485427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:44.498560 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae353cd4-bf61-4c7d-801b-02fd68de57cd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:44.517754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:44.558308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:44.558935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:44.559219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:44.559495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] PATH_INFO: `/v3/projects/2de80cacfcf14d6f946daa8b57826f30` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:44.560638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de80cacfcf14d6f946daa8b57826f30)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:44.562565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:44.608761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de80cacfcf14d6f946daa8b57826f30', 'request_id': 'req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.614220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de80cacfcf14d6f946daa8b57826f30', 'request_id': 'req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.622372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de80cacfcf14d6f946daa8b57826f30', 'request_id': 'req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.634631 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] The token cache is being invalidate because project 2de80cacfcf14d6f946daa8b57826f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:44.634998 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.635645 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e3d0dc4c-4ec6-438b-993a-c03127e4c4e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:44.637178 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1314/2629] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:44 2022] DELETE /identity/v3/projects/2de80cacfcf14d6f946daa8b57826f30 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:44.642111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ce3c-aef7-48ab-bfe6-14376d2a9c99 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:44.642581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ce3c-aef7-48ab-bfe6-14376d2a9c99 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:44.642678 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ce3c-aef7-48ab-bfe6-14376d2a9c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:44.664594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4d49ce3c-aef7-48ab-bfe6-14376d2a9c99 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:44.665951 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4d49ce3c-aef7-48ab-bfe6-14376d2a9c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:44.733314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1316/2630] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:44.741692 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c3d67-2b4f-4634-a05e-c27608f6a3f3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:44.742005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c3d67-2b4f-4634-a05e-c27608f6a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:44.742258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c3d67-2b4f-4634-a05e-c27608f6a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:44.762379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-175c3d67-2b4f-4634-a05e-c27608f6a3f3 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:44.763598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-175c3d67-2b4f-4634-a05e-c27608f6a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:44.772020 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1315/2631] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:44.776016 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49ce3c-aef7-48ab-bfe6-14376d2a9c99 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:44.786542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49ce3c-aef7-48ab-bfe6-14376d2a9c99 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:44.791173 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6000d5-e0b9-4ee1-bf09-90be2a258777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:44.791772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6000d5-e0b9-4ee1-bf09-90be2a258777 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:44.792080 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6000d5-e0b9-4ee1-bf09-90be2a258777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:44.792355 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6000d5-e0b9-4ee1-bf09-90be2a258777 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:44.793028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6000d5-e0b9-4ee1-bf09-90be2a258777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:30:44.793334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6000d5-e0b9-4ee1-bf09-90be2a258777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:44.794947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6000d5-e0b9-4ee1-bf09-90be2a258777 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:44.807541 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1317/2632] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:30:44.811766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c3d67-2b4f-4634-a05e-c27608f6a3f3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:44.821513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c3d67-2b4f-4634-a05e-c27608f6a3f3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:44.826145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3e9cf-84cd-4047-b068-e1046f4a569a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:44.826732 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3e9cf-84cd-4047-b068-e1046f4a569a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:44.827012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3e9cf-84cd-4047-b068-e1046f4a569a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:44.827285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3e9cf-84cd-4047-b068-e1046f4a569a admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:44.827916 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3e9cf-84cd-4047-b068-e1046f4a569a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:44.829893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3e9cf-84cd-4047-b068-e1046f4a569a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:44.857743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1316/2633] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:44 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:44.862323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6000d5-e0b9-4ee1-bf09-90be2a258777 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:44.873340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6000d5-e0b9-4ee1-bf09-90be2a258777 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:44.877885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:44.936898 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d092e-ad8a-4a22-b2a8-b1c5201bb20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:44.937493 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d092e-ad8a-4a22-b2a8-b1c5201bb20f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:44.937767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d092e-ad8a-4a22-b2a8-b1c5201bb20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:44.938034 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d092e-ad8a-4a22-b2a8-b1c5201bb20f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:44.938641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d092e-ad8a-4a22-b2a8-b1c5201bb20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:44.940586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d092e-ad8a-4a22-b2a8-b1c5201bb20f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:44.964275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1318/2634] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:44 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:44.965807 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3e9cf-84cd-4047-b068-e1046f4a569a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:44.976475 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3e9cf-84cd-4047-b068-e1046f4a569a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:44.981056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03002f77-73a2-48c8-818a-2e137c47f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:44.981675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03002f77-73a2-48c8-818a-2e137c47f62a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:44.981953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03002f77-73a2-48c8-818a-2e137c47f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:44.982223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03002f77-73a2-48c8-818a-2e137c47f62a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:44.982868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03002f77-73a2-48c8-818a-2e137c47f62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:44.984607 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03002f77-73a2-48c8-818a-2e137c47f62a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:44.995241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1317/2635] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:44.999226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d092e-ad8a-4a22-b2a8-b1c5201bb20f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.009528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d092e-ad8a-4a22-b2a8-b1c5201bb20f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.014117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5933620-9b7f-4e42-926b-2e9f7ea14cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.014707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5933620-9b7f-4e42-926b-2e9f7ea14cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.014990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5933620-9b7f-4e42-926b-2e9f7ea14cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.015306 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5933620-9b7f-4e42-926b-2e9f7ea14cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.015954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5933620-9b7f-4e42-926b-2e9f7ea14cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:45.017600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5933620-9b7f-4e42-926b-2e9f7ea14cbe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:45.028109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1319/2636] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:45.032210 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03002f77-73a2-48c8-818a-2e137c47f62a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.042899 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03002f77-73a2-48c8-818a-2e137c47f62a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.047449 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c91c3-a96d-4322-a4e6-0796aadce1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.048079 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c91c3-a96d-4322-a4e6-0796aadce1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.048356 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c91c3-a96d-4322-a4e6-0796aadce1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.048654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c91c3-a96d-4322-a4e6-0796aadce1b7 admin admin] PATH_INFO: `/v3/projects/90dee5ea255646b0ae87953a2deb0567/users/9a501705a8794165a204f39265f590d1/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.063284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c91c3-a96d-4322-a4e6-0796aadce1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90dee5ea255646b0ae87953a2deb0567, user_id=9a501705a8794165a204f39265f590d1, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:45.065965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c91c3-a96d-4322-a4e6-0796aadce1b7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:45.079017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1318/2637] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:45 2022] PUT /identity/v3/projects/90dee5ea255646b0ae87953a2deb0567/users/9a501705a8794165a204f39265f590d1/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:45.090149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5225-3399-4034-8aee-709408397439 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.090467 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5225-3399-4034-8aee-709408397439 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.090746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d5225-3399-4034-8aee-709408397439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.122707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0a7d5225-3399-4034-8aee-709408397439 None None] MFA Rules not processed for user `9a501705a8794165a204f39265f590d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:45.124359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a7d5225-3399-4034-8aee-709408397439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:45.192272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1320/2638] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:45.196798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c91c3-a96d-4322-a4e6-0796aadce1b7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.245402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:45.260602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c91c3-a96d-4322-a4e6-0796aadce1b7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.265185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b256ac-5cfa-4e1b-b2b0-10a15e4e6fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.265776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b256ac-5cfa-4e1b-b2b0-10a15e4e6fba admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.266056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b256ac-5cfa-4e1b-b2b0-10a15e4e6fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.266325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b256ac-5cfa-4e1b-b2b0-10a15e4e6fba admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.266952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b256ac-5cfa-4e1b-b2b0-10a15e4e6fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:45.268882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b256ac-5cfa-4e1b-b2b0-10a15e4e6fba admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:45.309722 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1319/2639] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:45 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:45.310962 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d5225-3399-4034-8aee-709408397439 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.322224 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d5225-3399-4034-8aee-709408397439 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.385563 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c490c-549a-4b10-90cb-ee71392391c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.386103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c490c-549a-4b10-90cb-ee71392391c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.386385 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c490c-549a-4b10-90cb-ee71392391c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.386661 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c490c-549a-4b10-90cb-ee71392391c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.387298 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c490c-549a-4b10-90cb-ee71392391c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:45.389281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554c490c-549a-4b10-90cb-ee71392391c4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:45.413381 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1321/2640] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:45 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:30:45.417484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b256ac-5cfa-4e1b-b2b0-10a15e4e6fba admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.428005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b256ac-5cfa-4e1b-b2b0-10a15e4e6fba admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.432877 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359e7c1-ac08-4e48-b405-f57f23774e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.433476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359e7c1-ac08-4e48-b405-f57f23774e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.433829 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359e7c1-ac08-4e48-b405-f57f23774e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.434094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359e7c1-ac08-4e48-b405-f57f23774e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.434740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359e7c1-ac08-4e48-b405-f57f23774e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:45.436409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359e7c1-ac08-4e48-b405-f57f23774e2c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:45.447036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1320/2641] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:45.450967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c490c-549a-4b10-90cb-ee71392391c4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.461549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c490c-549a-4b10-90cb-ee71392391c4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.466122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fad79a-cb28-4e41-aab5-5fe6ff04d720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.466764 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fad79a-cb28-4e41-aab5-5fe6ff04d720 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.467043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fad79a-cb28-4e41-aab5-5fe6ff04d720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.467317 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fad79a-cb28-4e41-aab5-5fe6ff04d720 admin admin] PATH_INFO: `/v3/projects/20b21d79c9be42d5a1195f29549dfa6a/users/d78ac0a34cac4bfd867c27a9b11d175e/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.469772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:45.484938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fad79a-cb28-4e41-aab5-5fe6ff04d720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b21d79c9be42d5a1195f29549dfa6a, user_id=d78ac0a34cac4bfd867c27a9b11d175e, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:45.487609 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fad79a-cb28-4e41-aab5-5fe6ff04d720 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:45.500517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1322/2642] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:45 2022] PUT /identity/v3/projects/20b21d79c9be42d5a1195f29549dfa6a/users/d78ac0a34cac4bfd867c27a9b11d175e/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:30:45.505196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359e7c1-ac08-4e48-b405-f57f23774e2c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.515023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359e7c1-ac08-4e48-b405-f57f23774e2c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.573346 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d535031-70e7-40e4-80c3-f05abd192e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.573975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535031-70e7-40e4-80c3-f05abd192e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.574280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535031-70e7-40e4-80c3-f05abd192e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.574552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535031-70e7-40e4-80c3-f05abd192e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.575205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d535031-70e7-40e4-80c3-f05abd192e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:45.577008 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d535031-70e7-40e4-80c3-f05abd192e70 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:45.587499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1321/2643] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:45.591626 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fad79a-cb28-4e41-aab5-5fe6ff04d720 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.602413 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fad79a-cb28-4e41-aab5-5fe6ff04d720 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.607029 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2b3c8-1243-4d88-9b38-6963e0eb7816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.607671 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2b3c8-1243-4d88-9b38-6963e0eb7816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.607949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2b3c8-1243-4d88-9b38-6963e0eb7816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.608218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2b3c8-1243-4d88-9b38-6963e0eb7816 admin admin] PATH_INFO: `/v3/projects/20b21d79c9be42d5a1195f29549dfa6a/users/d78ac0a34cac4bfd867c27a9b11d175e/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.610146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2b3c8-1243-4d88-9b38-6963e0eb7816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b21d79c9be42d5a1195f29549dfa6a, user_id=d78ac0a34cac4bfd867c27a9b11d175e, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:45.612822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2b3c8-1243-4d88-9b38-6963e0eb7816 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:45.625124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1323/2644] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:45 2022] PUT /identity/v3/projects/20b21d79c9be42d5a1195f29549dfa6a/users/d78ac0a34cac4bfd867c27a9b11d175e/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:45.636915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ed92-34a1-49a2-9c64-62ef6485d0ac None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.637115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ed92-34a1-49a2-9c64-62ef6485d0ac None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.637430 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949ed92-34a1-49a2-9c64-62ef6485d0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.668059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3949ed92-34a1-49a2-9c64-62ef6485d0ac None None] MFA Rules not processed for user `d78ac0a34cac4bfd867c27a9b11d175e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:45.669308 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3949ed92-34a1-49a2-9c64-62ef6485d0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:45.746160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1322/2645] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:45 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:45.757126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2b3c8-1243-4d88-9b38-6963e0eb7816 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.758982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-51c2b3c8-1243-4d88-9b38-6963e0eb7816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:45.826615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2b3c8-1243-4d88-9b38-6963e0eb7816 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.830964 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cf4ba-d5bb-487d-bc8e-8d0fa92eefa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.831602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cf4ba-d5bb-487d-bc8e-8d0fa92eefa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.831880 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cf4ba-d5bb-487d-bc8e-8d0fa92eefa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.832160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cf4ba-d5bb-487d-bc8e-8d0fa92eefa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.842883 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cf4ba-d5bb-487d-bc8e-8d0fa92eefa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:45.844324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cf4ba-d5bb-487d-bc8e-8d0fa92eefa4 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:45.854767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1324/2646] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:45 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:45.906496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949ed92-34a1-49a2-9c64-62ef6485d0ac None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:45.907902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3949ed92-34a1-49a2-9c64-62ef6485d0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:45.922704 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949ed92-34a1-49a2-9c64-62ef6485d0ac None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:45.927628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccb4dc-4f1f-4720-a37b-ae09c1b1c8a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:45.928256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccb4dc-4f1f-4720-a37b-ae09c1b1c8a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:45.928584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccb4dc-4f1f-4720-a37b-ae09c1b1c8a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:45.929061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccb4dc-4f1f-4720-a37b-ae09c1b1c8a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:45.985665 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:46.000047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccb4dc-4f1f-4720-a37b-ae09c1b1c8a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:46.001566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccb4dc-4f1f-4720-a37b-ae09c1b1c8a3 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:46.011355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1323/2647] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:49.360296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cf4ba-d5bb-487d-bc8e-8d0fa92eefa4 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:49.430002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cf4ba-d5bb-487d-bc8e-8d0fa92eefa4 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:49.434637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:49.435312 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:49.435591 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:49.435863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] PATH_INFO: `/v3/users/9a501705a8794165a204f39265f590d1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:49.437024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a501705a8794165a204f39265f590d1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:49.439020 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:49.485296 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a501705a8794165a204f39265f590d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:49.486152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a501705a8794165a204f39265f590d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:49.486740 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a501705a8794165a204f39265f590d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:49.494206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a501705a8794165a204f39265f590d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:49.507061 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a501705a8794165a204f39265f590d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:49.508744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:49.521180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1325/2648] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:49 2022] DELETE /identity/v3/users/9a501705a8794165a204f39265f590d1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:49.525685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09efa5-734b-41e2-973a-bc426095ff8f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:49.526165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09efa5-734b-41e2-973a-bc426095ff8f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:49.526249 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09efa5-734b-41e2-973a-bc426095ff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:49.547719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ee09efa5-734b-41e2-973a-bc426095ff8f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:49.548940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ee09efa5-734b-41e2-973a-bc426095ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:49.607632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1324/2649] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:49.617597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:49.618986 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:49.683331 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac3c8b-4c26-478c-8f70-a45aac55dcaf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:49.686946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d0258-55aa-43c9-b212-64d057c8ce0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:49.687556 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d0258-55aa-43c9-b212-64d057c8ce0d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:49.687834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d0258-55aa-43c9-b212-64d057c8ce0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:49.688107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d0258-55aa-43c9-b212-64d057c8ce0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:49.698094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d0258-55aa-43c9-b212-64d057c8ce0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:49.699525 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d0258-55aa-43c9-b212-64d057c8ce0d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:49.709237 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1326/2650] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:50.030042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09efa5-734b-41e2-973a-bc426095ff8f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:50.031717 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ee09efa5-734b-41e2-973a-bc426095ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:50.047314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09efa5-734b-41e2-973a-bc426095ff8f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:50.052947 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:50.053595 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:50.053883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:50.054711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] PATH_INFO: `/v3/projects/90dee5ea255646b0ae87953a2deb0567` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:50.056057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90dee5ea255646b0ae87953a2deb0567)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:50.058036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:50.108377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90dee5ea255646b0ae87953a2deb0567', 'request_id': 'req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.114366 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90dee5ea255646b0ae87953a2deb0567', 'request_id': 'req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.124055 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90dee5ea255646b0ae87953a2deb0567', 'request_id': 'req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.139764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] The token cache is being invalidate because project 90dee5ea255646b0ae87953a2deb0567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:50.140076 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.140787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7fd412fd-e2dd-4b9b-b549-e8dd0ed3e478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.142299 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1325/2651] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:50 2022] DELETE /identity/v3/projects/90dee5ea255646b0ae87953a2deb0567 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:50.147479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d0258-55aa-43c9-b212-64d057c8ce0d service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:50.157783 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d0258-55aa-43c9-b212-64d057c8ce0d service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:50.217254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:50.217947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:50.218150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:50.218490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] PATH_INFO: `/v3/users/d78ac0a34cac4bfd867c27a9b11d175e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:50.219676 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78ac0a34cac4bfd867c27a9b11d175e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:50.221645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:50.271348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78ac0a34cac4bfd867c27a9b11d175e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.272087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78ac0a34cac4bfd867c27a9b11d175e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.272693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78ac0a34cac4bfd867c27a9b11d175e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.279718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78ac0a34cac4bfd867c27a9b11d175e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.293281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96ac3fc3-0749-4a6c-addc-830e6f4cf5e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78ac0a34cac4bfd867c27a9b11d175e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.307328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1327/2652] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:50 2022] DELETE /identity/v3/users/d78ac0a34cac4bfd867c27a9b11d175e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:50.485137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185b744-3cc5-438e-a6ba-82cc527821d0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:50.485394 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185b744-3cc5-438e-a6ba-82cc527821d0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:50.485683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185b744-3cc5-438e-a6ba-82cc527821d0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:50.486980 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1326/2653] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:30:50.493868 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd04ac-778e-4119-bb2d-d4a6633fab2e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:50.494354 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd04ac-778e-4119-bb2d-d4a6633fab2e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:50.494474 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd04ac-778e-4119-bb2d-d4a6633fab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:50.526802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-89cd04ac-778e-4119-bb2d-d4a6633fab2e None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:50.528418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-89cd04ac-778e-4119-bb2d-d4a6633fab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:50.539911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:50.637773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1328/2654] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:50.651946 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185b744-3cc5-438e-a6ba-82cc527821d0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:50.655044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9185b744-3cc5-438e-a6ba-82cc527821d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:50.715827 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd04ac-778e-4119-bb2d-d4a6633fab2e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:50.717564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-89cd04ac-778e-4119-bb2d-d4a6633fab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:50.774730 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185b744-3cc5-438e-a6ba-82cc527821d0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:50.780419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98273a2d-388b-441a-9390-20fedeff2a94 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de0dfcbad22348fdaea32d64557e68a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:50.781116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98273a2d-388b-441a-9390-20fedeff2a94 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:50.781495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98273a2d-388b-441a-9390-20fedeff2a94 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:50.781839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98273a2d-388b-441a-9390-20fedeff2a94 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:50.794825 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98273a2d-388b-441a-9390-20fedeff2a94 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:50.796698 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98273a2d-388b-441a-9390-20fedeff2a94 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:50.799718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd04ac-778e-4119-bb2d-d4a6633fab2e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:50.806269 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:50.807115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:50.807483 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:50.807853 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] PATH_INFO: `/v3/projects/20b21d79c9be42d5a1195f29549dfa6a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:50.809164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b21d79c9be42d5a1195f29549dfa6a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:50.811597 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:50.824901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1327/2655] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:50.934196 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a37099-9e15-437d-bc13-a81c7660503c None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:50.934852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a37099-9e15-437d-bc13-a81c7660503c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:50.935308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b21d79c9be42d5a1195f29549dfa6a', 'request_id': 'req-f569449d-7e62-4202-8ef5-958b17f4c69c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.935466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a37099-9e15-437d-bc13-a81c7660503c None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:50.937040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1328/2656] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:30:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:30:50.942296 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f81a30-d21a-426b-94ee-1937937f1b3d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:50.942575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f81a30-d21a-426b-94ee-1937937f1b3d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:50.942857 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f81a30-d21a-426b-94ee-1937937f1b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:50.944281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b21d79c9be42d5a1195f29549dfa6a', 'request_id': 'req-f569449d-7e62-4202-8ef5-958b17f4c69c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.957757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b21d79c9be42d5a1195f29549dfa6a', 'request_id': 'req-f569449d-7e62-4202-8ef5-958b17f4c69c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.966637 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-18f81a30-d21a-426b-94ee-1937937f1b3d None None] MFA Rules not processed for user `c8704aae85084d41bdedb5537ef96e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:50.968039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-18f81a30-d21a-426b-94ee-1937937f1b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:50.971014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] The token cache is being invalidate because project 20b21d79c9be42d5a1195f29549dfa6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:50.971311 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.971960 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f569449d-7e62-4202-8ef5-958b17f4c69c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:50.974406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1329/2657] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:50 2022] DELETE /identity/v3/projects/20b21d79c9be42d5a1195f29549dfa6a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:30:50.975139 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:50.979705 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d4c1d-6a42-46c6-9515-3727bf6eca71 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:50.980835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d4c1d-6a42-46c6-9515-3727bf6eca71 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:50.981997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d4c1d-6a42-46c6-9515-3727bf6eca71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.008628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-927d4c1d-6a42-46c6-9515-3727bf6eca71 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:51.009899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-927d4c1d-6a42-46c6-9515-3727bf6eca71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:51.040693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1329/2658] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:51.055205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f81a30-d21a-426b-94ee-1937937f1b3d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.056793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:51.057025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-18f81a30-d21a-426b-94ee-1937937f1b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:51.069906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f81a30-d21a-426b-94ee-1937937f1b3d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.076031 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1feca-a7b1-495b-820b-25bf571fb000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.076972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1feca-a7b1-495b-820b-25bf571fb000 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.077218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1feca-a7b1-495b-820b-25bf571fb000 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.077489 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1feca-a7b1-495b-820b-25bf571fb000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.086872 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1feca-a7b1-495b-820b-25bf571fb000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.088349 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1feca-a7b1-495b-820b-25bf571fb000 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.093959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1330/2659] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:51.096754 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1330/2660] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:30:51.102217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6236489-2e19-48a4-8d80-041ecea0a219 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.102481 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6236489-2e19-48a4-8d80-041ecea0a219 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.102815 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6236489-2e19-48a4-8d80-041ecea0a219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.126195 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c6236489-2e19-48a4-8d80-041ecea0a219 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:51.127817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c6236489-2e19-48a4-8d80-041ecea0a219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:51.138651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1331/2661] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:51.142937 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1feca-a7b1-495b-820b-25bf571fb000 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.158003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1feca-a7b1-495b-820b-25bf571fb000 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.164411 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680b3ea-47a7-43b7-825d-342495e65272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.165109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680b3ea-47a7-43b7-825d-342495e65272 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.165465 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680b3ea-47a7-43b7-825d-342495e65272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.165810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680b3ea-47a7-43b7-825d-342495e65272 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.166533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680b3ea-47a7-43b7-825d-342495e65272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:30:51.166911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680b3ea-47a7-43b7-825d-342495e65272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.168782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680b3ea-47a7-43b7-825d-342495e65272 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.184440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1331/2662] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:30:51.186782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6236489-2e19-48a4-8d80-041ecea0a219 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.188027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbe209-14ae-4cca-bfad-29ec4aae71b0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.188407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbe209-14ae-4cca-bfad-29ec4aae71b0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.188786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbe209-14ae-4cca-bfad-29ec4aae71b0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.190192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1332/2663] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 15:30:51.196071 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31209cb-8361-4de0-be6c-131e836adf77 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.196446 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31209cb-8361-4de0-be6c-131e836adf77 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.196576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6236489-2e19-48a4-8d80-041ecea0a219 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.196855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31209cb-8361-4de0-be6c-131e836adf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.201242 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23745e24-c526-4358-90ad-eea3818d497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.201842 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23745e24-c526-4358-90ad-eea3818d497a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.202119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23745e24-c526-4358-90ad-eea3818d497a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.202388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23745e24-c526-4358-90ad-eea3818d497a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.203012 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23745e24-c526-4358-90ad-eea3818d497a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.204940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23745e24-c526-4358-90ad-eea3818d497a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.217456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d31209cb-8361-4de0-be6c-131e836adf77 None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:51.218601 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d31209cb-8361-4de0-be6c-131e836adf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:51.234881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1332/2664] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:51 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:51.238991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23745e24-c526-4358-90ad-eea3818d497a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.303894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1333/2665] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:51.307754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23745e24-c526-4358-90ad-eea3818d497a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.312586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1c23b-e672-4fe1-aa2c-fc7797ea9c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.313285 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1c23b-e672-4fe1-aa2c-fc7797ea9c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.313640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1c23b-e672-4fe1-aa2c-fc7797ea9c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.313984 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1c23b-e672-4fe1-aa2c-fc7797ea9c32 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.314660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1c23b-e672-4fe1-aa2c-fc7797ea9c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.315909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31209cb-8361-4de0-be6c-131e836adf77 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.316681 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1c23b-e672-4fe1-aa2c-fc7797ea9c32 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.326258 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31209cb-8361-4de0-be6c-131e836adf77 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.341048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1333/2666] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:51 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:30:51.345227 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1c23b-e672-4fe1-aa2c-fc7797ea9c32 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.357459 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1c23b-e672-4fe1-aa2c-fc7797ea9c32 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.362209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275a0b-5b80-450b-8e7c-11a09b41a3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.362834 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275a0b-5b80-450b-8e7c-11a09b41a3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.363167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275a0b-5b80-450b-8e7c-11a09b41a3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.363509 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99275a0b-5b80-450b-8e7c-11a09b41a3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.364197 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99275a0b-5b80-450b-8e7c-11a09b41a3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.365948 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99275a0b-5b80-450b-8e7c-11a09b41a3e1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.378890 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1334/2667] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:51.383086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275a0b-5b80-450b-8e7c-11a09b41a3e1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.389969 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248795e-920a-428d-a8a2-a1055edc7eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.390572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248795e-920a-428d-a8a2-a1055edc7eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.390855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248795e-920a-428d-a8a2-a1055edc7eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.391149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248795e-920a-428d-a8a2-a1055edc7eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.393364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99275a0b-5b80-450b-8e7c-11a09b41a3e1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.399063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645dfea-26d1-44b5-99d0-e18c256aea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.399655 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645dfea-26d1-44b5-99d0-e18c256aea97 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.399994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645dfea-26d1-44b5-99d0-e18c256aea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.400292 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645dfea-26d1-44b5-99d0-e18c256aea97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.401021 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645dfea-26d1-44b5-99d0-e18c256aea97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.402472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645dfea-26d1-44b5-99d0-e18c256aea97 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.414478 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1335/2668] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:51.421698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645dfea-26d1-44b5-99d0-e18c256aea97 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.435528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645dfea-26d1-44b5-99d0-e18c256aea97 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.439871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:51.440573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916c163-6d0b-4cb3-abeb-f1f7d674af18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.441295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916c163-6d0b-4cb3-abeb-f1f7d674af18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.441712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916c163-6d0b-4cb3-abeb-f1f7d674af18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.441831 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916c163-6d0b-4cb3-abeb-f1f7d674af18 admin admin] PATH_INFO: `/v3/projects/6a00f95fee09433c9625d6f43ff4ff27/users/724bb6e1513848b99ae512c535b361a6/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.456815 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916c163-6d0b-4cb3-abeb-f1f7d674af18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a00f95fee09433c9625d6f43ff4ff27, user_id=724bb6e1513848b99ae512c535b361a6, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.460044 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916c163-6d0b-4cb3-abeb-f1f7d674af18 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.472164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248795e-920a-428d-a8a2-a1055edc7eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.474090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1336/2669] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:51 2022] PUT /identity/v3/projects/6a00f95fee09433c9625d6f43ff4ff27/users/724bb6e1513848b99ae512c535b361a6/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:51.474390 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248795e-920a-428d-a8a2-a1055edc7eb6 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.485263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eba986-3a22-4299-b415-87d21f9fee47 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.485664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eba986-3a22-4299-b415-87d21f9fee47 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.486107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eba986-3a22-4299-b415-87d21f9fee47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.486256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1334/2670] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:30:51.515662 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-99eba986-3a22-4299-b415-87d21f9fee47 None None] MFA Rules not processed for user `724bb6e1513848b99ae512c535b361a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:51.516912 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-99eba986-3a22-4299-b415-87d21f9fee47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:51.557833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:51.589031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1337/2671] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:51.593710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248795e-920a-428d-a8a2-a1055edc7eb6 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.605518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248795e-920a-428d-a8a2-a1055edc7eb6 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.667799 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a67ea-fee7-4bca-9330-a76943c94df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.668574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a67ea-fee7-4bca-9330-a76943c94df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.668853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a67ea-fee7-4bca-9330-a76943c94df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.669168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a67ea-fee7-4bca-9330-a76943c94df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.669954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a67ea-fee7-4bca-9330-a76943c94df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.672798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a67ea-fee7-4bca-9330-a76943c94df5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.702497 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1335/2672] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:51 2022] POST /identity/v3/projects => generated 399 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:51.706945 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eba986-3a22-4299-b415-87d21f9fee47 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.724927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eba986-3a22-4299-b415-87d21f9fee47 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.805932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226f290-7d6c-437b-8a59-3da4297e4034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.806605 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226f290-7d6c-437b-8a59-3da4297e4034 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.806889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226f290-7d6c-437b-8a59-3da4297e4034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.807160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226f290-7d6c-437b-8a59-3da4297e4034 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.807795 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226f290-7d6c-437b-8a59-3da4297e4034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.809781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226f290-7d6c-437b-8a59-3da4297e4034 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.837498 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1338/2673] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:51 2022] POST /identity/v3/users => generated 306 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:51.841793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a67ea-fee7-4bca-9330-a76943c94df5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.852559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a67ea-fee7-4bca-9330-a76943c94df5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.858170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cd632-412d-46db-b4d6-44d71bff8a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.858785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cd632-412d-46db-b4d6-44d71bff8a9a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.859066 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cd632-412d-46db-b4d6-44d71bff8a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.859339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cd632-412d-46db-b4d6-44d71bff8a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.859996 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cd632-412d-46db-b4d6-44d71bff8a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.861864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cd632-412d-46db-b4d6-44d71bff8a9a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.883472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1336/2674] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:51.888098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226f290-7d6c-437b-8a59-3da4297e4034 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.902707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226f290-7d6c-437b-8a59-3da4297e4034 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:51.914533 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438e1d1-41c7-4a38-a417-b19ebb2dbb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:51.916742 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438e1d1-41c7-4a38-a417-b19ebb2dbb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:51.917033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438e1d1-41c7-4a38-a417-b19ebb2dbb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:51.917316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438e1d1-41c7-4a38-a417-b19ebb2dbb0a admin admin] PATH_INFO: `/v3/projects/8a5add426c5743fb9e93ffef530bc11b/users/d2ab89655c974178882733b5bb9324e4/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:51.940504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438e1d1-41c7-4a38-a417-b19ebb2dbb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5add426c5743fb9e93ffef530bc11b, user_id=d2ab89655c974178882733b5bb9324e4, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:51.941779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438e1d1-41c7-4a38-a417-b19ebb2dbb0a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:51.958054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1339/2675] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:51 2022] PUT /identity/v3/projects/8a5add426c5743fb9e93ffef530bc11b/users/d2ab89655c974178882733b5bb9324e4/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:51.962083 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cd632-412d-46db-b4d6-44d71bff8a9a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:51.974204 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cd632-412d-46db-b4d6-44d71bff8a9a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:52.036696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b1ae3-f27f-426b-9375-a6f0a0deec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:52.037301 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b1ae3-f27f-426b-9375-a6f0a0deec17 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:52.037582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b1ae3-f27f-426b-9375-a6f0a0deec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:52.037854 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b1ae3-f27f-426b-9375-a6f0a0deec17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:52.038496 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b1ae3-f27f-426b-9375-a6f0a0deec17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:52.040111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b1ae3-f27f-426b-9375-a6f0a0deec17 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:52.050451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1337/2676] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:52.055163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438e1d1-41c7-4a38-a417-b19ebb2dbb0a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:52.068154 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438e1d1-41c7-4a38-a417-b19ebb2dbb0a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:52.073682 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ac943-bc37-4af4-9492-be2d7b5c2a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:52.074337 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ac943-bc37-4af4-9492-be2d7b5c2a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:52.074611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ac943-bc37-4af4-9492-be2d7b5c2a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:52.074887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ac943-bc37-4af4-9492-be2d7b5c2a09 admin admin] PATH_INFO: `/v3/projects/8a5add426c5743fb9e93ffef530bc11b/users/d2ab89655c974178882733b5bb9324e4/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:52.077422 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ac943-bc37-4af4-9492-be2d7b5c2a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5add426c5743fb9e93ffef530bc11b, user_id=d2ab89655c974178882733b5bb9324e4, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:52.080256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ac943-bc37-4af4-9492-be2d7b5c2a09 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:52.092348 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1340/2677] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:52 2022] PUT /identity/v3/projects/8a5add426c5743fb9e93ffef530bc11b/users/d2ab89655c974178882733b5bb9324e4/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:52.161985 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2b06c-8ac9-4b53-b3c0-481868ff084d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:52.162305 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2b06c-8ac9-4b53-b3c0-481868ff084d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:52.162582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2b06c-8ac9-4b53-b3c0-481868ff084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:52.190555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ebe2b06c-8ac9-4b53-b3c0-481868ff084d None None] MFA Rules not processed for user `d2ab89655c974178882733b5bb9324e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:52.192002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ebe2b06c-8ac9-4b53-b3c0-481868ff084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:52.216325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:52.264547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1338/2678] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:52 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:52.276112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ac943-bc37-4af4-9492-be2d7b5c2a09 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:52.277767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ce8ac943-bc37-4af4-9492-be2d7b5c2a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:52.342026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ac943-bc37-4af4-9492-be2d7b5c2a09 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:52.345872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25569fbc-f94e-473f-96d7-fb3c16008306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:52.346472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25569fbc-f94e-473f-96d7-fb3c16008306 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:52.346744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25569fbc-f94e-473f-96d7-fb3c16008306 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:52.347006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25569fbc-f94e-473f-96d7-fb3c16008306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:52.356843 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25569fbc-f94e-473f-96d7-fb3c16008306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:52.358448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25569fbc-f94e-473f-96d7-fb3c16008306 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:52.368841 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1341/2679] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:52 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:52.446313 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2b06c-8ac9-4b53-b3c0-481868ff084d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:52.456061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2b06c-8ac9-4b53-b3c0-481868ff084d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:52.460558 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c43386-3563-48a9-87b3-608c4a24341d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:52.461183 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c43386-3563-48a9-87b3-608c4a24341d service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:52.461463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c43386-3563-48a9-87b3-608c4a24341d service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:52.461734 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c43386-3563-48a9-87b3-608c4a24341d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:52.528462 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c43386-3563-48a9-87b3-608c4a24341d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:52.529836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c43386-3563-48a9-87b3-608c4a24341d service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:52.538566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1339/2680] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:52 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:53.875902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc7f12-8d19-4ce9-9b1a-4109c5aa0bcf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:53.876444 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc7f12-8d19-4ce9-9b1a-4109c5aa0bcf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:53.876544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc7f12-8d19-4ce9-9b1a-4109c5aa0bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:53.896954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c3bc7f12-8d19-4ce9-9b1a-4109c5aa0bcf None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:53.899006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c3bc7f12-8d19-4ce9-9b1a-4109c5aa0bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:53.938932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:53.966600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1342/2681] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:30:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:53.981421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c43386-3563-48a9-87b3-608c4a24341d service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:53.982846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a8c43386-3563-48a9-87b3-608c4a24341d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:54.072286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c43386-3563-48a9-87b3-608c4a24341d service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:54.077510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf551c0f-8dbb-49f0-8209-2612dfa22de9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:54.078269 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf551c0f-8dbb-49f0-8209-2612dfa22de9 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:54.078548 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf551c0f-8dbb-49f0-8209-2612dfa22de9 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:54.078821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf551c0f-8dbb-49f0-8209-2612dfa22de9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:54.088643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf551c0f-8dbb-49f0-8209-2612dfa22de9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:54.090253 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf551c0f-8dbb-49f0-8209-2612dfa22de9 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:54.098553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1340/2682] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:30:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:30:56.829710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc7f12-8d19-4ce9-9b1a-4109c5aa0bcf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:56.940559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc7f12-8d19-4ce9-9b1a-4109c5aa0bcf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:56.946672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:56.947346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:56.947620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:56.948141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] PATH_INFO: `/v3/users/724bb6e1513848b99ae512c535b361a6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:56.949368 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724bb6e1513848b99ae512c535b361a6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:56.951441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:57.006013 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724bb6e1513848b99ae512c535b361a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.006663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724bb6e1513848b99ae512c535b361a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.007265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724bb6e1513848b99ae512c535b361a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.014727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724bb6e1513848b99ae512c535b361a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.027182 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724bb6e1513848b99ae512c535b361a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.048367 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1343/2683] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:56 2022] DELETE /identity/v3/users/724bb6e1513848b99ae512c535b361a6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:57.054044 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601ec9f-17a8-44db-af67-db49d9213e05 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:57.054539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601ec9f-17a8-44db-af67-db49d9213e05 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:57.054715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601ec9f-17a8-44db-af67-db49d9213e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:57.076628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2601ec9f-17a8-44db-af67-db49d9213e05 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:57.077944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2601ec9f-17a8-44db-af67-db49d9213e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:57.131107 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:57.150225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1341/2684] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:57.161581 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:57.166187 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:57.240508 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83922d9f-52dc-420d-a829-9a9015e81b54 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:57.244417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7f232-a0db-4591-aec5-a1a7ad914527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:57.245041 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7f232-a0db-4591-aec5-a1a7ad914527 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:57.245323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7f232-a0db-4591-aec5-a1a7ad914527 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:57.245598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7f232-a0db-4591-aec5-a1a7ad914527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:57.257648 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7f232-a0db-4591-aec5-a1a7ad914527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:57.259295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7f232-a0db-4591-aec5-a1a7ad914527 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:57.270813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1344/2685] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:30:57.570964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601ec9f-17a8-44db-af67-db49d9213e05 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:57.572499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2601ec9f-17a8-44db-af67-db49d9213e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:57.586865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601ec9f-17a8-44db-af67-db49d9213e05 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:57.592214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:57.593015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:57.593392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:57.593780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] PATH_INFO: `/v3/projects/6a00f95fee09433c9625d6f43ff4ff27` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:57.595259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a00f95fee09433c9625d6f43ff4ff27)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:57.597551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:57.655223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a00f95fee09433c9625d6f43ff4ff27', 'request_id': 'req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.662633 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a00f95fee09433c9625d6f43ff4ff27', 'request_id': 'req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.672468 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a00f95fee09433c9625d6f43ff4ff27', 'request_id': 'req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.701981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] The token cache is being invalidate because project 6a00f95fee09433c9625d6f43ff4ff27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:57.701981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.701981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.701981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1342/2686] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:57 2022] DELETE /identity/v3/projects/6a00f95fee09433c9625d6f43ff4ff27 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:57.702454 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7f232-a0db-4591-aec5-a1a7ad914527 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:57.704608 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:57.710958 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7f232-a0db-4591-aec5-a1a7ad914527 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:57.788967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:57.789717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:57.790073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:57.790420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] PATH_INFO: `/v3/users/d2ab89655c974178882733b5bb9324e4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:57.791684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ab89655c974178882733b5bb9324e4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:57.793956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:57.853731 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ab89655c974178882733b5bb9324e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.854927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ab89655c974178882733b5bb9324e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.855606 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ab89655c974178882733b5bb9324e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.862965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ab89655c974178882733b5bb9324e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.876863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-95e1475f-b2cb-4a55-b12d-6802653dda85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ab89655c974178882733b5bb9324e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:57.893432 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1345/2687] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:30:57 2022] DELETE /identity/v3/users/d2ab89655c974178882733b5bb9324e4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:58.364748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:58.366790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:58.400867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0ec3b-d13c-4b3c-8b36-635f71412cb9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:58.478952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:58.479508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:58.479873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:58.480152 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] PATH_INFO: `/v3/projects/8a5add426c5743fb9e93ffef530bc11b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:58.481365 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5add426c5743fb9e93ffef530bc11b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:58.484235 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:58.550710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5add426c5743fb9e93ffef530bc11b', 'request_id': 'req-49c06081-befb-4528-aba0-374e4498e1a7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:58.565555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5add426c5743fb9e93ffef530bc11b', 'request_id': 'req-49c06081-befb-4528-aba0-374e4498e1a7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:58.577564 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5add426c5743fb9e93ffef530bc11b', 'request_id': 'req-49c06081-befb-4528-aba0-374e4498e1a7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:58.590801 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] The token cache is being invalidate because project 8a5add426c5743fb9e93ffef530bc11b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:30:58.591109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:58.591790 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-49c06081-befb-4528-aba0-374e4498e1a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:30:58.593407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1343/2688] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:30:58 2022] DELETE /identity/v3/projects/8a5add426c5743fb9e93ffef530bc11b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:58.598406 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f4b3c-a8da-41c0-b021-158e70188c64 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:58.598763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f4b3c-a8da-41c0-b021-158e70188c64 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:58.599160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f4b3c-a8da-41c0-b021-158e70188c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:58.624879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9e7f4b3c-a8da-41c0-b021-158e70188c64 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:58.626325 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e7f4b3c-a8da-41c0-b021-158e70188c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:58.704726 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1346/2689] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:58.712758 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ec24f-69bb-4bb9-9f44-3406aeaf7137 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:58.713271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ec24f-69bb-4bb9-9f44-3406aeaf7137 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:58.713709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ec24f-69bb-4bb9-9f44-3406aeaf7137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:58.721974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:58.736972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a79ec24f-69bb-4bb9-9f44-3406aeaf7137 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:58.739202 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a79ec24f-69bb-4bb9-9f44-3406aeaf7137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:58.752755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1344/2690] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:30:58.758434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f4b3c-a8da-41c0-b021-158e70188c64 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:58.768611 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f4b3c-a8da-41c0-b021-158e70188c64 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:58.773933 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237011e-e822-458c-9fc0-0b9037f8d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:58.774555 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237011e-e822-458c-9fc0-0b9037f8d14a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:58.774838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237011e-e822-458c-9fc0-0b9037f8d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:58.775115 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237011e-e822-458c-9fc0-0b9037f8d14a admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:58.775776 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237011e-e822-458c-9fc0-0b9037f8d14a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:30:58.776086 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237011e-e822-458c-9fc0-0b9037f8d14a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:58.777763 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237011e-e822-458c-9fc0-0b9037f8d14a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:58.790711 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1347/2691] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:30:58.795278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ec24f-69bb-4bb9-9f44-3406aeaf7137 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:58.805772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ec24f-69bb-4bb9-9f44-3406aeaf7137 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:58.810644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e125a-d8e1-4653-aa51-aa703200b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:58.811245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e125a-d8e1-4653-aa51-aa703200b99a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:58.811553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e125a-d8e1-4653-aa51-aa703200b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:58.811826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e125a-d8e1-4653-aa51-aa703200b99a admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:58.812494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e125a-d8e1-4653-aa51-aa703200b99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:58.814421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e125a-d8e1-4653-aa51-aa703200b99a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:58.848654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1345/2692] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:30:58 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:58.852767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237011e-e822-458c-9fc0-0b9037f8d14a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:58.863455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237011e-e822-458c-9fc0-0b9037f8d14a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:58.868441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:30:58.939342 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a156b-fbff-4262-9bad-95230e1d2033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:58.939965 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a156b-fbff-4262-9bad-95230e1d2033 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:58.940492 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a156b-fbff-4262-9bad-95230e1d2033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:58.940782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a156b-fbff-4262-9bad-95230e1d2033 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:58.942116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a156b-fbff-4262-9bad-95230e1d2033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:58.945112 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a156b-fbff-4262-9bad-95230e1d2033 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:58.972210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1348/2693] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:30:58 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:30:58.976924 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e125a-d8e1-4653-aa51-aa703200b99a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:58.990279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e125a-d8e1-4653-aa51-aa703200b99a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:58.996289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9e400-ac22-463d-8f83-349e9fe7d749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:58.997547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9e400-ac22-463d-8f83-349e9fe7d749 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:58.997896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9e400-ac22-463d-8f83-349e9fe7d749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:58.998100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9e400-ac22-463d-8f83-349e9fe7d749 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:58.999114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9e400-ac22-463d-8f83-349e9fe7d749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:59.001169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9e400-ac22-463d-8f83-349e9fe7d749 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:59.020154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1346/2694] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:30:59.021384 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a156b-fbff-4262-9bad-95230e1d2033 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:59.032791 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a156b-fbff-4262-9bad-95230e1d2033 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:59.037792 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5474b99-98ae-4a99-ae28-3f79858446c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:59.038415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5474b99-98ae-4a99-ae28-3f79858446c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:59.038691 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5474b99-98ae-4a99-ae28-3f79858446c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:59.038967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5474b99-98ae-4a99-ae28-3f79858446c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:59.040159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5474b99-98ae-4a99-ae28-3f79858446c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:59.041840 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5474b99-98ae-4a99-ae28-3f79858446c6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:59.055650 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1349/2695] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:30:59.060047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9e400-ac22-463d-8f83-349e9fe7d749 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:59.085685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9e400-ac22-463d-8f83-349e9fe7d749 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:59.091102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed412e7-cbc2-4a53-ba87-871a5fd0b74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:59.091766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed412e7-cbc2-4a53-ba87-871a5fd0b74c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:59.092046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed412e7-cbc2-4a53-ba87-871a5fd0b74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:59.092319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed412e7-cbc2-4a53-ba87-871a5fd0b74c admin admin] PATH_INFO: `/v3/projects/7111b041a72b4481b3c8a286589e0a4e/users/55a7b30c3c094a8088c370a0d79ac98a/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:59.109251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed412e7-cbc2-4a53-ba87-871a5fd0b74c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7111b041a72b4481b3c8a286589e0a4e, user_id=55a7b30c3c094a8088c370a0d79ac98a, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:59.112002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed412e7-cbc2-4a53-ba87-871a5fd0b74c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:59.139287 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1347/2696] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:30:59 2022] PUT /identity/v3/projects/7111b041a72b4481b3c8a286589e0a4e/users/55a7b30c3c094a8088c370a0d79ac98a/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:30:59.145527 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accd2fb-711e-4ec0-8c30-8cc81af666ec None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:59.150569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accd2fb-711e-4ec0-8c30-8cc81af666ec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:59.150975 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accd2fb-711e-4ec0-8c30-8cc81af666ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:59.172857 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9accd2fb-711e-4ec0-8c30-8cc81af666ec None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:30:59.174437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9accd2fb-711e-4ec0-8c30-8cc81af666ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:59.259575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1350/2697] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:30:59.267711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed412e7-cbc2-4a53-ba87-871a5fd0b74c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:30:59.269951 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1ed412e7-cbc2-4a53-ba87-871a5fd0b74c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:30:59.364759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed412e7-cbc2-4a53-ba87-871a5fd0b74c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:30:59.366524 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68436ba-abe9-423d-baff-57a9aa3d5447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:30:59.367162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68436ba-abe9-423d-baff-57a9aa3d5447 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:30:59.367434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68436ba-abe9-423d-baff-57a9aa3d5447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:30:59.367711 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68436ba-abe9-423d-baff-57a9aa3d5447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:30:59.389538 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68436ba-abe9-423d-baff-57a9aa3d5447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:30:59.394154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68436ba-abe9-423d-baff-57a9aa3d5447 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:30:59.416354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1348/2698] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:31:04.601918 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b49a91-d309-4db8-86b7-5e8bbba37244 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:04.602520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b49a91-d309-4db8-86b7-5e8bbba37244 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:04.602520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b49a91-d309-4db8-86b7-5e8bbba37244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:04.651750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b9b49a91-d309-4db8-86b7-5e8bbba37244 None None] MFA Rules not processed for user `55a7b30c3c094a8088c370a0d79ac98a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:31:04.653067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b9b49a91-d309-4db8-86b7-5e8bbba37244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:04.757490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:04.772618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1351/2699] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:31:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:31:04.776748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68436ba-abe9-423d-baff-57a9aa3d5447 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:04.837113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68436ba-abe9-423d-baff-57a9aa3d5447 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:04.841756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff1d1e-604c-4269-a50f-306a81a6131c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:04.842363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff1d1e-604c-4269-a50f-306a81a6131c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:04.842641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff1d1e-604c-4269-a50f-306a81a6131c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:04.842909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff1d1e-604c-4269-a50f-306a81a6131c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:04.843535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff1d1e-604c-4269-a50f-306a81a6131c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:04.845511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff1d1e-604c-4269-a50f-306a81a6131c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:04.889445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1349/2700] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:31:04 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:31:04.892051 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b49a91-d309-4db8-86b7-5e8bbba37244 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:04.903949 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b49a91-d309-4db8-86b7-5e8bbba37244 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:04.964326 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ede623-2cf7-4643-a03a-0659ca2fa9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:04.964967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ede623-2cf7-4643-a03a-0659ca2fa9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:04.965243 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ede623-2cf7-4643-a03a-0659ca2fa9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:04.965510 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ede623-2cf7-4643-a03a-0659ca2fa9e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:04.966146 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ede623-2cf7-4643-a03a-0659ca2fa9e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:04.968169 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ede623-2cf7-4643-a03a-0659ca2fa9e7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:04.991469 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1352/2701] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:31:04 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:31:04.995437 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff1d1e-604c-4269-a50f-306a81a6131c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:05.025328 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff1d1e-604c-4269-a50f-306a81a6131c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:05.030057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08b41b-4dc8-4e20-a163-90d9a67e7870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:05.030756 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08b41b-4dc8-4e20-a163-90d9a67e7870 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:05.031033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08b41b-4dc8-4e20-a163-90d9a67e7870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:05.031300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08b41b-4dc8-4e20-a163-90d9a67e7870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:05.031941 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08b41b-4dc8-4e20-a163-90d9a67e7870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:05.033700 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08b41b-4dc8-4e20-a163-90d9a67e7870 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:05.044119 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1350/2702] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:31:05.048281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ede623-2cf7-4643-a03a-0659ca2fa9e7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:05.060441 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ede623-2cf7-4643-a03a-0659ca2fa9e7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:05.068075 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e22fd-6d30-48a7-9c0c-506a312aa470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:05.068772 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e22fd-6d30-48a7-9c0c-506a312aa470 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:05.069047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e22fd-6d30-48a7-9c0c-506a312aa470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:05.069320 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e22fd-6d30-48a7-9c0c-506a312aa470 admin admin] PATH_INFO: `/v3/projects/8977fcec1a794f1494a07cc3e0344b68/users/686f6293fd7248c59cc05475215b2f30/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:05.085549 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e22fd-6d30-48a7-9c0c-506a312aa470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8977fcec1a794f1494a07cc3e0344b68, user_id=686f6293fd7248c59cc05475215b2f30, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:05.088240 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e22fd-6d30-48a7-9c0c-506a312aa470 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:05.105470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1353/2703] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:31:05 2022] PUT /identity/v3/projects/8977fcec1a794f1494a07cc3e0344b68/users/686f6293fd7248c59cc05475215b2f30/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:31:05.109151 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08b41b-4dc8-4e20-a163-90d9a67e7870 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:05.119302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08b41b-4dc8-4e20-a163-90d9a67e7870 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:05.133824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:05.188455 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c01749-51e0-43df-a109-f4fe2cdb981b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:05.189025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01749-51e0-43df-a109-f4fe2cdb981b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:05.189323 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01749-51e0-43df-a109-f4fe2cdb981b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:05.189578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c01749-51e0-43df-a109-f4fe2cdb981b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:05.190179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01749-51e0-43df-a109-f4fe2cdb981b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:05.192445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c01749-51e0-43df-a109-f4fe2cdb981b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:05.203227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1351/2704] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:31:05.207911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e22fd-6d30-48a7-9c0c-506a312aa470 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:05.221387 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e22fd-6d30-48a7-9c0c-506a312aa470 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:05.227282 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea8d0f-1a34-484d-805f-1d0265fc4e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:05.228063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea8d0f-1a34-484d-805f-1d0265fc4e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:05.228438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea8d0f-1a34-484d-805f-1d0265fc4e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:05.228784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea8d0f-1a34-484d-805f-1d0265fc4e98 admin admin] PATH_INFO: `/v3/projects/8977fcec1a794f1494a07cc3e0344b68/users/686f6293fd7248c59cc05475215b2f30/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:05.230879 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea8d0f-1a34-484d-805f-1d0265fc4e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8977fcec1a794f1494a07cc3e0344b68, user_id=686f6293fd7248c59cc05475215b2f30, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:05.233662 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea8d0f-1a34-484d-805f-1d0265fc4e98 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:05.245453 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1354/2705] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:31:05 2022] PUT /identity/v3/projects/8977fcec1a794f1494a07cc3e0344b68/users/686f6293fd7248c59cc05475215b2f30/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:31:10.055725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52207a18-a4ba-47c4-9e7b-160f3ac36c80 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:10.056660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52207a18-a4ba-47c4-9e7b-160f3ac36c80 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:10.059135 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52207a18-a4ba-47c4-9e7b-160f3ac36c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:10.095125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-52207a18-a4ba-47c4-9e7b-160f3ac36c80 None None] MFA Rules not processed for user `686f6293fd7248c59cc05475215b2f30`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:31:10.096641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-52207a18-a4ba-47c4-9e7b-160f3ac36c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:10.197170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1352/2706] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:31:10.212899 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea8d0f-1a34-484d-805f-1d0265fc4e98 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:10.214896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88ea8d0f-1a34-484d-805f-1d0265fc4e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:10.402314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:10.414004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea8d0f-1a34-484d-805f-1d0265fc4e98 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:10.420415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972b1ee-61c3-4ada-b76a-aae5e7babf37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:10.421331 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972b1ee-61c3-4ada-b76a-aae5e7babf37 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:10.421629 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972b1ee-61c3-4ada-b76a-aae5e7babf37 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:10.421908 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972b1ee-61c3-4ada-b76a-aae5e7babf37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:10.489732 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972b1ee-61c3-4ada-b76a-aae5e7babf37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:10.491334 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972b1ee-61c3-4ada-b76a-aae5e7babf37 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:10.502649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1355/2707] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:31:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:31:11.986027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5de2dc-d147-4e77-9243-3ba6a3064a83 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:11.986500 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5de2dc-d147-4e77-9243-3ba6a3064a83 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:11.986598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5de2dc-d147-4e77-9243-3ba6a3064a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:12.007059 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3f5de2dc-d147-4e77-9243-3ba6a3064a83 None None] MFA Rules not processed for user `55a7b30c3c094a8088c370a0d79ac98a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:31:12.008290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f5de2dc-d147-4e77-9243-3ba6a3064a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:12.015352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1353/2708] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:31:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:31:12.103587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972b1ee-61c3-4ada-b76a-aae5e7babf37 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:12.105048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b972b1ee-61c3-4ada-b76a-aae5e7babf37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:12.213470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972b1ee-61c3-4ada-b76a-aae5e7babf37 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:12.220024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de9296-e3d0-4679-a856-2dff45787b43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:12.220747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de9296-e3d0-4679-a856-2dff45787b43 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:12.221122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de9296-e3d0-4679-a856-2dff45787b43 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:12.221463 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de9296-e3d0-4679-a856-2dff45787b43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:12.224087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a9de9296-e3d0-4679-a856-2dff45787b43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:12.246811 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de9296-e3d0-4679-a856-2dff45787b43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:12.248683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de9296-e3d0-4679-a856-2dff45787b43 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:12.261750 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1356/2709] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:31:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:31:12.613068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5de2dc-d147-4e77-9243-3ba6a3064a83 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:12.614818 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f5de2dc-d147-4e77-9243-3ba6a3064a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:12.693628 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5de2dc-d147-4e77-9243-3ba6a3064a83 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:12.698226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:12.698870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:12.699154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:12.699440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] PATH_INFO: `/v3/users/d348fd6ae41b4112b8dad30a907738b3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:12.700606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d348fd6ae41b4112b8dad30a907738b3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:12.702557 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:12.745540 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:12.762592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd348fd6ae41b4112b8dad30a907738b3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:12.774819 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd348fd6ae41b4112b8dad30a907738b3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:12.775553 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd348fd6ae41b4112b8dad30a907738b3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:12.791111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd348fd6ae41b4112b8dad30a907738b3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:12.799506 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd348fd6ae41b4112b8dad30a907738b3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:12.809330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1354/2710] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:31:12 2022] DELETE /identity/v3/users/d348fd6ae41b4112b8dad30a907738b3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:31:12.813859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e89928-b294-4138-9386-8efc1be14b7f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:12.814214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e89928-b294-4138-9386-8efc1be14b7f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:12.814499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e89928-b294-4138-9386-8efc1be14b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:12.848863 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-64e89928-b294-4138-9386-8efc1be14b7f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:31:12.850263 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-64e89928-b294-4138-9386-8efc1be14b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:12.925602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1357/2711] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:31:12.937470 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:12.940776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:13.026271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bc7db-3dad-4796-8f91-a8c1aec68635 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:13.030000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dc580-d04d-4ca1-8c5f-8f677ace2b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:13.030954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dc580-d04d-4ca1-8c5f-8f677ace2b7a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:13.031300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dc580-d04d-4ca1-8c5f-8f677ace2b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:13.031652 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dc580-d04d-4ca1-8c5f-8f677ace2b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:13.042285 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0dc580-d04d-4ca1-8c5f-8f677ace2b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:13.043839 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0dc580-d04d-4ca1-8c5f-8f677ace2b7a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:13.058082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1355/2712] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:31:13.390994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e89928-b294-4138-9386-8efc1be14b7f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:13.392896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-64e89928-b294-4138-9386-8efc1be14b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:13.404033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:13.420279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e89928-b294-4138-9386-8efc1be14b7f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:13.432996 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:13.433706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:13.433991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:13.434264 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] PATH_INFO: `/v3/projects/d2ba9fce1d2948d3b7ab9cea5d936ea4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:13.437079 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ba9fce1d2948d3b7ab9cea5d936ea4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:13.439431 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:13.553014 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ba9fce1d2948d3b7ab9cea5d936ea4', 'request_id': 'req-af496023-9a7e-40c2-96a7-c77db5086ffd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:13.598219 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ba9fce1d2948d3b7ab9cea5d936ea4', 'request_id': 'req-af496023-9a7e-40c2-96a7-c77db5086ffd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:13.610957 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ba9fce1d2948d3b7ab9cea5d936ea4', 'request_id': 'req-af496023-9a7e-40c2-96a7-c77db5086ffd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:13.635885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] The token cache is being invalidate because project d2ba9fce1d2948d3b7ab9cea5d936ea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:31:13.636233 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:13.636907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:31:13.648600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1358/2713] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:31:13 2022] DELETE /identity/v3/projects/d2ba9fce1d2948d3b7ab9cea5d936ea4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:31:15.046382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58345ac-53a8-4b4e-9d0a-bb913678718c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:15.046914 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58345ac-53a8-4b4e-9d0a-bb913678718c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:15.047068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58345ac-53a8-4b4e-9d0a-bb913678718c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:15.080580 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d58345ac-53a8-4b4e-9d0a-bb913678718c None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:31:15.081098 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d58345ac-53a8-4b4e-9d0a-bb913678718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:15.145840 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1356/2714] 10.222.0.75 () {58 vars in 967 bytes} [Fri Aug 19 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:31:15.158254 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:15.159663 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:15.235346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af496023-9a7e-40c2-96a7-c77db5086ffd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:15.241172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4ebfa-fc7b-42ac-8905-36668af98311 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:15.241794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4ebfa-fc7b-42ac-8905-36668af98311 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:15.242072 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4ebfa-fc7b-42ac-8905-36668af98311 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:15.242345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4ebfa-fc7b-42ac-8905-36668af98311 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:15.251094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4ebfa-fc7b-42ac-8905-36668af98311 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:15.252765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4ebfa-fc7b-42ac-8905-36668af98311 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:15.267126 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1359/2715] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:31:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:31:17.987391 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58345ac-53a8-4b4e-9d0a-bb913678718c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:17.988885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d58345ac-53a8-4b4e-9d0a-bb913678718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:18.050936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58345ac-53a8-4b4e-9d0a-bb913678718c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:18.054852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:18.055463 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:18.055742 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:18.056012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:18.057812 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:18.062785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:18.119225 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:18.120713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:18.131273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1357/2716] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:31:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:31:25.260692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:26.041150 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2135f-283c-46f6-8ada-87bed3d1716c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:26.041430 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2135f-283c-46f6-8ada-87bed3d1716c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:26.041702 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2135f-283c-46f6-8ada-87bed3d1716c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:26.061501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c0c2135f-283c-46f6-8ada-87bed3d1716c None None] MFA Rules not processed for user `af3e7eea0b2840b79404b370382ab44a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:31:26.062744 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c0c2135f-283c-46f6-8ada-87bed3d1716c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:26.069907 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1360/2717] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:31:28.125804 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=25 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:36.057471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:45.717433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:45.718884 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:45.798765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337085f6-7144-46cd-93b7-3805bf0c4e9e service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:45.802451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f87ae9-b749-4440-b90a-726042adf499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:45.803325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f87ae9-b749-4440-b90a-726042adf499 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:45.803605 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f87ae9-b749-4440-b90a-726042adf499 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:45.803876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f87ae9-b749-4440-b90a-726042adf499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:45.806317 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-81f87ae9-b749-4440-b90a-726042adf499 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:45.873206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f87ae9-b749-4440-b90a-726042adf499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:45.874668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f87ae9-b749-4440-b90a-726042adf499 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:45.884170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1358/2718] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:31:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:31:48.116658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2135f-283c-46f6-8ada-87bed3d1716c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:31:48.127692 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2135f-283c-46f6-8ada-87bed3d1716c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:31:48.132189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ec85c-68cf-4143-9cac-2bc893f1e877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:31:48.133094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ec85c-68cf-4143-9cac-2bc893f1e877 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:31:48.133488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ec85c-68cf-4143-9cac-2bc893f1e877 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:31:48.134043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ec85c-68cf-4143-9cac-2bc893f1e877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:31:48.136583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a77ec85c-68cf-4143-9cac-2bc893f1e877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:31:48.211720 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ec85c-68cf-4143-9cac-2bc893f1e877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:31:48.214071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ec85c-68cf-4143-9cac-2bc893f1e877 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:31:48.225535 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1361/2719] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:31:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:31:55.879968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=59 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:31:58.220220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=30 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:32:04.808224 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89e36-ec24-45b4-b82a-316e15c82ea6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:32:04.808800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89e36-ec24-45b4-b82a-316e15c82ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:32:04.808800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d89e36-ec24-45b4-b82a-316e15c82ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:32:04.950005 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d1d89e36-ec24-45b4-b82a-316e15c82ea6 None None] MFA Rules not processed for user `821d38464cd3491e81e94d809930d076`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:32:04.951143 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d1d89e36-ec24-45b4-b82a-316e15c82ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:32:05.005133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1359/2720] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:32:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:32:14.998688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=25 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:33:02.659281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ec85c-68cf-4143-9cac-2bc893f1e877 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:02.668286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ec85c-68cf-4143-9cac-2bc893f1e877 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:02.673272 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65efd9f8-1b14-402d-b6a8-985aa887b376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:02.673889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efd9f8-1b14-402d-b6a8-985aa887b376 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:02.674164 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efd9f8-1b14-402d-b6a8-985aa887b376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:02.674436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efd9f8-1b14-402d-b6a8-985aa887b376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:02.676183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-65efd9f8-1b14-402d-b6a8-985aa887b376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:02.760110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65efd9f8-1b14-402d-b6a8-985aa887b376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:02.761415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65efd9f8-1b14-402d-b6a8-985aa887b376 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:02.773022 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1362/2721] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:33:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:33:06.310679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89e36-ec24-45b4-b82a-316e15c82ea6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:06.320927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d89e36-ec24-45b4-b82a-316e15c82ea6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:06.324434 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e1682-f0d9-47a8-a8f4-545a77c4dd10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:06.325027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e1682-f0d9-47a8-a8f4-545a77c4dd10 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:06.325306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e1682-f0d9-47a8-a8f4-545a77c4dd10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:06.325578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e1682-f0d9-47a8-a8f4-545a77c4dd10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:06.327891 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-519e1682-f0d9-47a8-a8f4-545a77c4dd10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:06.341109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e1682-f0d9-47a8-a8f4-545a77c4dd10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:06.342531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e1682-f0d9-47a8-a8f4-545a77c4dd10 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:06.353663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1360/2722] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:33:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:33:10.129747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85662cb3-86b2-4be9-94f3-baa7cf0172d5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:10.130281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85662cb3-86b2-4be9-94f3-baa7cf0172d5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:10.130281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85662cb3-86b2-4be9-94f3-baa7cf0172d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:10.150484 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-85662cb3-86b2-4be9-94f3-baa7cf0172d5 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:33:10.151687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-85662cb3-86b2-4be9-94f3-baa7cf0172d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:10.158982 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1363/2723] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:33:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:33:10.206074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e1682-f0d9-47a8-a8f4-545a77c4dd10 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:10.217195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e1682-f0d9-47a8-a8f4-545a77c4dd10 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:10.222214 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11313c-c574-4faf-8564-b75bb1ac6313 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:10.222802 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11313c-c574-4faf-8564-b75bb1ac6313 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:10.223082 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11313c-c574-4faf-8564-b75bb1ac6313 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:10.223350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11313c-c574-4faf-8564-b75bb1ac6313 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:10.231144 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11313c-c574-4faf-8564-b75bb1ac6313 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:10.232761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11313c-c574-4faf-8564-b75bb1ac6313 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:10.240939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1361/2724] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:33:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:33:20.145033 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=37 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:33:20.237727 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=12 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:33:26.033631 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78cb77-9088-4949-9ef2-0d4548c9c8b6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:26.034116 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78cb77-9088-4949-9ef2-0d4548c9c8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:26.034247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78cb77-9088-4949-9ef2-0d4548c9c8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:26.053289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fb78cb77-9088-4949-9ef2-0d4548c9c8b6 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:33:26.054485 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fb78cb77-9088-4949-9ef2-0d4548c9c8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:26.061808 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1364/2725] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:33:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:33:26.074168 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11313c-c574-4faf-8564-b75bb1ac6313 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:26.085177 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11313c-c574-4faf-8564-b75bb1ac6313 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:26.090100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae8322-8a72-45dd-aaa2-be5bcffb3cf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:26.090688 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae8322-8a72-45dd-aaa2-be5bcffb3cf0 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:26.090968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae8322-8a72-45dd-aaa2-be5bcffb3cf0 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:26.091244 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae8322-8a72-45dd-aaa2-be5bcffb3cf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:26.099352 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae8322-8a72-45dd-aaa2-be5bcffb3cf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:26.101041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae8322-8a72-45dd-aaa2-be5bcffb3cf0 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:26.108968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1362/2726] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:33:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:33:36.047884 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:33:36.105795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:33:41.303719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78cb77-9088-4949-9ef2-0d4548c9c8b6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:41.305873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fb78cb77-9088-4949-9ef2-0d4548c9c8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:41.322675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78cb77-9088-4949-9ef2-0d4548c9c8b6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:41.328788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518ffe6-2d31-4d34-8a79-ef9945763db6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:41.329407 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518ffe6-2d31-4d34-8a79-ef9945763db6 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:41.329767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518ffe6-2d31-4d34-8a79-ef9945763db6 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:41.330042 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518ffe6-2d31-4d34-8a79-ef9945763db6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:41.331856 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e518ffe6-2d31-4d34-8a79-ef9945763db6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:41.344944 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518ffe6-2d31-4d34-8a79-ef9945763db6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:41.346546 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518ffe6-2d31-4d34-8a79-ef9945763db6 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:41.357641 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1365/2727] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:33:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:33:45.787896 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aade9-d8ac-4f42-98f8-e290b824b7c7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:45.789146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aade9-d8ac-4f42-98f8-e290b824b7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:45.789490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4aade9-d8ac-4f42-98f8-e290b824b7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:45.842348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8a4aade9-d8ac-4f42-98f8-e290b824b7c7 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:33:45.843551 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8a4aade9-d8ac-4f42-98f8-e290b824b7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:45.861622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1363/2728] 10.222.0.75 () {58 vars in 967 bytes} [Fri Aug 19 15:33:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:33:45.874278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518ffe6-2d31-4d34-8a79-ef9945763db6 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:45.896289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518ffe6-2d31-4d34-8a79-ef9945763db6 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:45.901830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ba958-aa99-4ad5-8d34-76b81b89bace service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:45.902437 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ba958-aa99-4ad5-8d34-76b81b89bace service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:45.902718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ba958-aa99-4ad5-8d34-76b81b89bace service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:45.902996 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ba958-aa99-4ad5-8d34-76b81b89bace service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:45.912244 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ba958-aa99-4ad5-8d34-76b81b89bace service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:45.913881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ba958-aa99-4ad5-8d34-76b81b89bace service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:45.924104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1366/2729] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:33:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:33:50.979469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4aade9-d8ac-4f42-98f8-e290b824b7c7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:50.981101 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8a4aade9-d8ac-4f42-98f8-e290b824b7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:50.992926 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4aade9-d8ac-4f42-98f8-e290b824b7c7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:50.996415 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:50.997006 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:50.997288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:50.997554 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:50.999232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:51.062512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:51.064608 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:51.073989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1364/2730] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:33:55.919000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=16 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:33:57.245652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ba958-aa99-4ad5-8d34-76b81b89bace service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:57.256252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ba958-aa99-4ad5-8d34-76b81b89bace service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:57.260089 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c02840-4dd9-4b38-907d-a007405330aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:57.260874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c02840-4dd9-4b38-907d-a007405330aa service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:57.261160 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c02840-4dd9-4b38-907d-a007405330aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:57.261435 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c02840-4dd9-4b38-907d-a007405330aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:57.263214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-17c02840-4dd9-4b38-907d-a007405330aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:57.273835 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c02840-4dd9-4b38-907d-a007405330aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:57.275813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c02840-4dd9-4b38-907d-a007405330aa service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:57.292847 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1367/2731] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:33:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:33:57.505372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:57.507417 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:57.523984 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a2a56-8caa-471f-b003-57173cbc6556 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:57.529289 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:57.530041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:57.530574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:57.531064 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] PATH_INFO: `/v3/users/af3e7eea0b2840b79404b370382ab44a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:57.532793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3e7eea0b2840b79404b370382ab44a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:57.535583 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:57.623382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3e7eea0b2840b79404b370382ab44a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:57.632229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3e7eea0b2840b79404b370382ab44a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:57.632987 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3e7eea0b2840b79404b370382ab44a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:57.650190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3e7eea0b2840b79404b370382ab44a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:57.659659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3e7eea0b2840b79404b370382ab44a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:57.683729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1365/2732] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:33:57 2022] DELETE /identity/v3/users/af3e7eea0b2840b79404b370382ab44a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:33:57.918129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c02840-4dd9-4b38-907d-a007405330aa service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:57.933349 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c02840-4dd9-4b38-907d-a007405330aa service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:57.992703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:57.993333 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:57.993612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:57.993886 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] PATH_INFO: `/v3/projects/0add00db6668475093f558e9700333d1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:57.995026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0add00db6668475093f558e9700333d1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:57.996993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:58.042943 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0add00db6668475093f558e9700333d1', 'request_id': 'req-f6ca796d-8995-45eb-9a16-bd756e356ebf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.048895 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0add00db6668475093f558e9700333d1', 'request_id': 'req-f6ca796d-8995-45eb-9a16-bd756e356ebf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.056499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0add00db6668475093f558e9700333d1', 'request_id': 'req-f6ca796d-8995-45eb-9a16-bd756e356ebf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.069565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] The token cache is being invalidate because project 0add00db6668475093f558e9700333d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:33:58.069849 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.070470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.071904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1368/2733] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:33:57 2022] DELETE /identity/v3/projects/0add00db6668475093f558e9700333d1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:33:58.075940 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:58.077355 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:58.095920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7270ea9-973c-4b95-9205-35c1f26e21f1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:58.124820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:33:58.159721 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:58.160345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:58.160648 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:58.160918 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] PATH_INFO: `/v3/users/12e33953d0414c13ab92a40c7cddb7b8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:58.162077 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e33953d0414c13ab92a40c7cddb7b8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:58.163984 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:58.210923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e33953d0414c13ab92a40c7cddb7b8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.217684 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e33953d0414c13ab92a40c7cddb7b8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.218341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e33953d0414c13ab92a40c7cddb7b8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.230621 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e33953d0414c13ab92a40c7cddb7b8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.238318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6b34674d-79ed-4cad-a3b3-9a260e7d017d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e33953d0414c13ab92a40c7cddb7b8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.244337 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1366/2734] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:33:58 2022] DELETE /identity/v3/users/12e33953d0414c13ab92a40c7cddb7b8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:33:58.421784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:33:58.423275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:33:58.436304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca796d-8995-45eb-9a16-bd756e356ebf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:33:58.495090 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:33:58.495645 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:33:58.495917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:33:58.496183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] PATH_INFO: `/v3/projects/7c817ad50ec0407b81e579cfd37a47a4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:33:58.497313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c817ad50ec0407b81e579cfd37a47a4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:33:58.499231 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:33:58.520426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:33:58.547184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c817ad50ec0407b81e579cfd37a47a4', 'request_id': 'req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.553001 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c817ad50ec0407b81e579cfd37a47a4', 'request_id': 'req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.560409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c817ad50ec0407b81e579cfd37a47a4', 'request_id': 'req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.571361 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] The token cache is being invalidate because project 7c817ad50ec0407b81e579cfd37a47a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:33:58.571675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.572302 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f869905f-4e12-48f0-8ab0-6bc0cf18a30d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:33:58.573830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1369/2735] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:33:58 2022] DELETE /identity/v3/projects/7c817ad50ec0407b81e579cfd37a47a4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:00.272569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9a50a-eea8-4975-95ab-343b4654e158 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:00.273145 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9a50a-eea8-4975-95ab-343b4654e158 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:00.273267 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9a50a-eea8-4975-95ab-343b4654e158 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:00.274614 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1367/2736] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:34:00.280615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639277e-cbe0-4fac-8575-267a2d06ef7d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:00.281109 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639277e-cbe0-4fac-8575-267a2d06ef7d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:00.281200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639277e-cbe0-4fac-8575-267a2d06ef7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:00.334687 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e639277e-cbe0-4fac-8575-267a2d06ef7d None None] MFA Rules not processed for user `c8704aae85084d41bdedb5537ef96e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:00.336021 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e639277e-cbe0-4fac-8575-267a2d06ef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:00.391698 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1370/2737] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:34:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:00.403826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9a50a-eea8-4975-95ab-343b4654e158 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:00.406481 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-99b9a50a-eea8-4975-95ab-343b4654e158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:00.419379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9a50a-eea8-4975-95ab-343b4654e158 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:00.423612 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bda72-1367-4ed7-9695-2d65f47cf26c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:00.424221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bda72-1367-4ed7-9695-2d65f47cf26c service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:00.424507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bda72-1367-4ed7-9695-2d65f47cf26c service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:00.424779 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bda72-1367-4ed7-9695-2d65f47cf26c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:00.431974 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bda72-1367-4ed7-9695-2d65f47cf26c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:00.433433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bda72-1367-4ed7-9695-2d65f47cf26c service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:00.441039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1368/2738] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:34:00.524218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d301-4741-4094-a6f1-5182779595b2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:00.524542 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d301-4741-4094-a6f1-5182779595b2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:00.524818 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d301-4741-4094-a6f1-5182779595b2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:00.526140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1371/2739] 10.222.0.75 () {54 vars in 921 bytes} [Fri Aug 19 15:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 15:34:00.532227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b3e17-741b-410d-a468-7fa1142cea31 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:00.532782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b3e17-741b-410d-a468-7fa1142cea31 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:00.533092 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b3e17-741b-410d-a468-7fa1142cea31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:00.570421 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7d3b3e17-741b-410d-a468-7fa1142cea31 None None] MFA Rules not processed for user `98a63d5b605a4ecfad2e305c491ae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:00.571610 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7d3b3e17-741b-410d-a468-7fa1142cea31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:00.638948 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1369/2740] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Aug 19 15:34:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:00.650434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d301-4741-4094-a6f1-5182779595b2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:00.652144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0400d301-4741-4094-a6f1-5182779595b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:00.665344 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d301-4741-4094-a6f1-5182779595b2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:00.668959 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15cd72-6653-4774-b5b1-6d99a6945eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:00.669568 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15cd72-6653-4774-b5b1-6d99a6945eed service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:00.669850 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15cd72-6653-4774-b5b1-6d99a6945eed service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:00.670124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15cd72-6653-4774-b5b1-6d99a6945eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:00.681688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15cd72-6653-4774-b5b1-6d99a6945eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:00.683147 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15cd72-6653-4774-b5b1-6d99a6945eed service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:00.694952 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1372/2741] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:00.759344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b3e17-741b-410d-a468-7fa1142cea31 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:00.760764 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7d3b3e17-741b-410d-a468-7fa1142cea31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:00.773590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:00.831820 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b3e17-741b-410d-a468-7fa1142cea31 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:00.837074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:00.837673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:00.837954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:00.838226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:00.839952 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:00.851870 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:00.853474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:00.863100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1370/2742] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:34:03.727614 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15cd72-6653-4774-b5b1-6d99a6945eed service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:03.729190 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7c15cd72-6653-4774-b5b1-6d99a6945eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:03.745855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15cd72-6653-4774-b5b1-6d99a6945eed service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:03.752037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d92ecd-9118-4273-b189-db3adccc2a4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98a63d5b605a4ecfad2e305c491ae755', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:03.752664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d92ecd-9118-4273-b189-db3adccc2a4d service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:03.752940 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d92ecd-9118-4273-b189-db3adccc2a4d service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:03.753209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d92ecd-9118-4273-b189-db3adccc2a4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:03.754946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-50d92ecd-9118-4273-b189-db3adccc2a4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:03.768729 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d92ecd-9118-4273-b189-db3adccc2a4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:03.770399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d92ecd-9118-4273-b189-db3adccc2a4d service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:03.782234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1373/2743] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:10.858860 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=27 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:13.776785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:20.269986 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:20.271403 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:20.349319 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267302b2-c121-476b-9b47-1d93efb6567b service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:20.352873 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:20.353464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:20.353741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:20.354035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:20.355728 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:20.412989 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:20.414450 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:20.423413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1371/2744] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:28.289620 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d92ecd-9118-4273-b189-db3adccc2a4d service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:28.300099 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d92ecd-9118-4273-b189-db3adccc2a4d service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:28.303598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734c487-958b-48ed-a999-dbac10592f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:28.304184 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734c487-958b-48ed-a999-dbac10592f4b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:28.304788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734c487-958b-48ed-a999-dbac10592f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:28.305055 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734c487-958b-48ed-a999-dbac10592f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:28.306794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5734c487-958b-48ed-a999-dbac10592f4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:28.370037 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734c487-958b-48ed-a999-dbac10592f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:28.371479 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734c487-958b-48ed-a999-dbac10592f4b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:28.381710 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1374/2745] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:30.419534 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=57 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:31.534832 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:31.536547 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:31.552555 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d81d51-8060-4e73-a1b4-701f38acfc65 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:31.557900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:31.558532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:31.561719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:31.562017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] PATH_INFO: `/v3/users/821d38464cd3491e81e94d809930d076` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:31.563320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=821d38464cd3491e81e94d809930d076)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:31.565976 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:31.627911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821d38464cd3491e81e94d809930d076'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:31.639155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821d38464cd3491e81e94d809930d076'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:31.640091 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821d38464cd3491e81e94d809930d076'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:31.660808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821d38464cd3491e81e94d809930d076'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:31.669578 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-748cb381-2a18-4c10-9b97-e06ff54a8105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821d38464cd3491e81e94d809930d076'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:31.678911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1372/2746] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:34:31 2022] DELETE /identity/v3/users/821d38464cd3491e81e94d809930d076 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:34:31.855251 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734c487-958b-48ed-a999-dbac10592f4b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:31.865929 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734c487-958b-48ed-a999-dbac10592f4b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:31.923682 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:31.924186 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:31.924501 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:31.924774 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] PATH_INFO: `/v3/projects/981bbb7282bd40d0b5335280d5041fa5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:31.925896 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=981bbb7282bd40d0b5335280d5041fa5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:31.927838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:31.974248 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '981bbb7282bd40d0b5335280d5041fa5', 'request_id': 'req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:31.979777 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '981bbb7282bd40d0b5335280d5041fa5', 'request_id': 'req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:31.986851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '981bbb7282bd40d0b5335280d5041fa5', 'request_id': 'req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:31.998253 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] The token cache is being invalidate because project 981bbb7282bd40d0b5335280d5041fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:34:31.998539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:31.999177 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-20dfbbdd-1e75-42b0-8f5e-e626178bfeb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:32.000675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1375/2747] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:34:31 2022] DELETE /identity/v3/projects/981bbb7282bd40d0b5335280d5041fa5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:32.006310 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b0f6-65ef-4f93-bce7-bf41a0bc28f8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.006594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b0f6-65ef-4f93-bce7-bf41a0bc28f8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.006867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b0f6-65ef-4f93-bce7-bf41a0bc28f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.047158 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8996b0f6-65ef-4f93-bce7-bf41a0bc28f8 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:32.048369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8996b0f6-65ef-4f93-bce7-bf41a0bc28f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:32.106894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1373/2748] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:32.115621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82722ac-c5a6-4d76-8645-71317a689c2f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.115906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82722ac-c5a6-4d76-8645-71317a689c2f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.116188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82722ac-c5a6-4d76-8645-71317a689c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.136308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e82722ac-c5a6-4d76-8645-71317a689c2f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:32.137860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e82722ac-c5a6-4d76-8645-71317a689c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:32.147596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1376/2749] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:32.151836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996b0f6-65ef-4f93-bce7-bf41a0bc28f8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.162149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996b0f6-65ef-4f93-bce7-bf41a0bc28f8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.166660 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c7775-4e3c-4a6c-9d9d-91a920a9452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.167238 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c7775-4e3c-4a6c-9d9d-91a920a9452a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.167515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c7775-4e3c-4a6c-9d9d-91a920a9452a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.167785 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c7775-4e3c-4a6c-9d9d-91a920a9452a admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.168458 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c7775-4e3c-4a6c-9d9d-91a920a9452a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:34:32.168762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c7775-4e3c-4a6c-9d9d-91a920a9452a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.170354 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c7775-4e3c-4a6c-9d9d-91a920a9452a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.183169 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1374/2750] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:34:32.188458 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82722ac-c5a6-4d76-8645-71317a689c2f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.199101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82722ac-c5a6-4d76-8645-71317a689c2f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.204328 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdc10a-f4c7-40ac-b27e-55d08b2b1bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.204987 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdc10a-f4c7-40ac-b27e-55d08b2b1bba admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.205830 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdc10a-f4c7-40ac-b27e-55d08b2b1bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.206113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdc10a-f4c7-40ac-b27e-55d08b2b1bba admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.206746 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdc10a-f4c7-40ac-b27e-55d08b2b1bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.208755 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdc10a-f4c7-40ac-b27e-55d08b2b1bba admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.224900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:32.247499 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1377/2751] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:34:32 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:32.251715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c7775-4e3c-4a6c-9d9d-91a920a9452a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.261782 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c7775-4e3c-4a6c-9d9d-91a920a9452a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.302499 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:32.318574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfcb20-71c7-40b4-adc9-a130adca690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.319160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfcb20-71c7-40b4-adc9-a130adca690b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.319436 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfcb20-71c7-40b4-adc9-a130adca690b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.319701 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfcb20-71c7-40b4-adc9-a130adca690b admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.320314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfcb20-71c7-40b4-adc9-a130adca690b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.322262 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfcb20-71c7-40b4-adc9-a130adca690b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.342965 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1375/2752] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:34:32 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:32.348158 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdc10a-f4c7-40ac-b27e-55d08b2b1bba admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.358546 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdc10a-f4c7-40ac-b27e-55d08b2b1bba admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.363782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932f5c3-9fe0-41b6-a0d4-9141ea167a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.364375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932f5c3-9fe0-41b6-a0d4-9141ea167a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.364686 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932f5c3-9fe0-41b6-a0d4-9141ea167a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.364961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932f5c3-9fe0-41b6-a0d4-9141ea167a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.365606 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932f5c3-9fe0-41b6-a0d4-9141ea167a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.367218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932f5c3-9fe0-41b6-a0d4-9141ea167a78 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.377917 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1378/2753] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:32.381908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfcb20-71c7-40b4-adc9-a130adca690b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.391482 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfcb20-71c7-40b4-adc9-a130adca690b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.396284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b0fc5-6f0a-4e9c-a70b-672d341e8adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.396748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0fc5-6f0a-4e9c-a70b-672d341e8adc admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.397023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0fc5-6f0a-4e9c-a70b-672d341e8adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.397292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0fc5-6f0a-4e9c-a70b-672d341e8adc admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.397919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b0fc5-6f0a-4e9c-a70b-672d341e8adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.399514 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b0fc5-6f0a-4e9c-a70b-672d341e8adc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.410527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1376/2754] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:32.414782 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932f5c3-9fe0-41b6-a0d4-9141ea167a78 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.425428 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932f5c3-9fe0-41b6-a0d4-9141ea167a78 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.430425 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c05e0-7c27-4e20-b09b-5ea852fd518f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.431141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c05e0-7c27-4e20-b09b-5ea852fd518f admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.431424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c05e0-7c27-4e20-b09b-5ea852fd518f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.431700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c05e0-7c27-4e20-b09b-5ea852fd518f admin admin] PATH_INFO: `/v3/projects/532218762bb84a10ae7bf94bde9147f9/users/2d04e5ce68cf48af9c5d5c73c5574181/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.446336 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c05e0-7c27-4e20-b09b-5ea852fd518f admin admin] RBAC: Authorizing `identity:create_grant(project_id=532218762bb84a10ae7bf94bde9147f9, user_id=2d04e5ce68cf48af9c5d5c73c5574181, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.449043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c05e0-7c27-4e20-b09b-5ea852fd518f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.461995 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1379/2755] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:34:32 2022] PUT /identity/v3/projects/532218762bb84a10ae7bf94bde9147f9/users/2d04e5ce68cf48af9c5d5c73c5574181/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:32.474512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e530c-bb46-47f6-b754-59e05fa33344 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.474795 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e530c-bb46-47f6-b754-59e05fa33344 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.475112 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e530c-bb46-47f6-b754-59e05fa33344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.506837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-420e530c-bb46-47f6-b754-59e05fa33344 None None] MFA Rules not processed for user `2d04e5ce68cf48af9c5d5c73c5574181`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:32.509048 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-420e530c-bb46-47f6-b754-59e05fa33344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:32.574720 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1377/2756] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:34:32.579210 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c05e0-7c27-4e20-b09b-5ea852fd518f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.639979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c05e0-7c27-4e20-b09b-5ea852fd518f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.644837 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c24f3-8dfb-4e1a-8849-9ccf5da2a89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.645438 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c24f3-8dfb-4e1a-8849-9ccf5da2a89b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.645723 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c24f3-8dfb-4e1a-8849-9ccf5da2a89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.646000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c24f3-8dfb-4e1a-8849-9ccf5da2a89b admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.646627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c24f3-8dfb-4e1a-8849-9ccf5da2a89b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.648588 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c24f3-8dfb-4e1a-8849-9ccf5da2a89b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.678992 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1380/2757] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:34:32 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:32.683251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e530c-bb46-47f6-b754-59e05fa33344 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.694456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e530c-bb46-47f6-b754-59e05fa33344 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.750533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7378d95-3f3c-4d3f-8bfd-f32192774add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.751053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7378d95-3f3c-4d3f-8bfd-f32192774add admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.751331 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7378d95-3f3c-4d3f-8bfd-f32192774add admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.751604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7378d95-3f3c-4d3f-8bfd-f32192774add admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.752786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7378d95-3f3c-4d3f-8bfd-f32192774add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.754703 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7378d95-3f3c-4d3f-8bfd-f32192774add admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.775945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1378/2758] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:34:32 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:32.780045 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c24f3-8dfb-4e1a-8849-9ccf5da2a89b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.790604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c24f3-8dfb-4e1a-8849-9ccf5da2a89b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.795284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b60527-589f-41a6-b93c-9bc02e819e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.795887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b60527-589f-41a6-b93c-9bc02e819e12 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.796163 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b60527-589f-41a6-b93c-9bc02e819e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.796476 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b60527-589f-41a6-b93c-9bc02e819e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.797122 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b60527-589f-41a6-b93c-9bc02e819e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.798741 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b60527-589f-41a6-b93c-9bc02e819e12 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.811114 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1381/2759] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:34:32.816036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7378d95-3f3c-4d3f-8bfd-f32192774add admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.825878 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7378d95-3f3c-4d3f-8bfd-f32192774add admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.830641 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e2aad-ba75-49e3-b098-5e0e76c0923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.831282 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e2aad-ba75-49e3-b098-5e0e76c0923d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.831565 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e2aad-ba75-49e3-b098-5e0e76c0923d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.831865 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e2aad-ba75-49e3-b098-5e0e76c0923d admin admin] PATH_INFO: `/v3/projects/481ad3299f024c1f9237c682bfb45ba2/users/bfd67c175eb84514b36033f652b60542/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.847363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e2aad-ba75-49e3-b098-5e0e76c0923d admin admin] RBAC: Authorizing `identity:create_grant(project_id=481ad3299f024c1f9237c682bfb45ba2, user_id=bfd67c175eb84514b36033f652b60542, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.850043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e2aad-ba75-49e3-b098-5e0e76c0923d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.863125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1379/2760] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:34:32 2022] PUT /identity/v3/projects/481ad3299f024c1f9237c682bfb45ba2/users/bfd67c175eb84514b36033f652b60542/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:32.867232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b60527-589f-41a6-b93c-9bc02e819e12 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.878054 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b60527-589f-41a6-b93c-9bc02e819e12 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.934104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc41e80-eeb5-4810-9d4a-4ade9aae7299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.934708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc41e80-eeb5-4810-9d4a-4ade9aae7299 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.934993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc41e80-eeb5-4810-9d4a-4ade9aae7299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.935266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc41e80-eeb5-4810-9d4a-4ade9aae7299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.935915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc41e80-eeb5-4810-9d4a-4ade9aae7299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.937612 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc41e80-eeb5-4810-9d4a-4ade9aae7299 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.947973 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1382/2761] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:32.952284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e2aad-ba75-49e3-b098-5e0e76c0923d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:32.963012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e2aad-ba75-49e3-b098-5e0e76c0923d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:32.967833 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5e859-5f41-46eb-845e-44a19e1ed6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:32.968505 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e859-5f41-46eb-845e-44a19e1ed6ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.968781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e859-5f41-46eb-845e-44a19e1ed6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.969049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5e859-5f41-46eb-845e-44a19e1ed6ab admin admin] PATH_INFO: `/v3/projects/481ad3299f024c1f9237c682bfb45ba2/users/bfd67c175eb84514b36033f652b60542/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:32.971632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5e859-5f41-46eb-845e-44a19e1ed6ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=481ad3299f024c1f9237c682bfb45ba2, user_id=bfd67c175eb84514b36033f652b60542, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:32.974373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5e859-5f41-46eb-845e-44a19e1ed6ab admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:32.984290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1380/2762] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:34:32 2022] PUT /identity/v3/projects/481ad3299f024c1f9237c682bfb45ba2/users/bfd67c175eb84514b36033f652b60542/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:32.994961 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ee154-9510-4be1-acea-befed13cdef2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:32.995252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ee154-9510-4be1-acea-befed13cdef2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:32.995590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ee154-9510-4be1-acea-befed13cdef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:33.023683 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a58ee154-9510-4be1-acea-befed13cdef2 None None] MFA Rules not processed for user `bfd67c175eb84514b36033f652b60542`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:33.024996 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a58ee154-9510-4be1-acea-befed13cdef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:33.034615 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:33.093468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1383/2763] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:33.105541 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5e859-5f41-46eb-845e-44a19e1ed6ab admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:33.107290 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-77a5e859-5f41-46eb-845e-44a19e1ed6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:33.114150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:33.166950 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5e859-5f41-46eb-845e-44a19e1ed6ab admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:33.170566 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2d86-602d-4769-99c0-b53d17acd253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:33.171162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2d86-602d-4769-99c0-b53d17acd253 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:33.171440 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2d86-602d-4769-99c0-b53d17acd253 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:33.171714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2d86-602d-4769-99c0-b53d17acd253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:33.181278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2d86-602d-4769-99c0-b53d17acd253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:33.182706 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2d86-602d-4769-99c0-b53d17acd253 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:33.191968 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1381/2764] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:33.237108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ee154-9510-4be1-acea-befed13cdef2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:33.238507 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a58ee154-9510-4be1-acea-befed13cdef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:33.250647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ee154-9510-4be1-acea-befed13cdef2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:33.254383 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d571d42-8e77-4b79-af32-80e4081c05b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:33.255005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571d42-8e77-4b79-af32-80e4081c05b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:33.255448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571d42-8e77-4b79-af32-80e4081c05b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:33.255728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571d42-8e77-4b79-af32-80e4081c05b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:33.310892 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d571d42-8e77-4b79-af32-80e4081c05b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:33.312261 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d571d42-8e77-4b79-af32-80e4081c05b7 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:33.321646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1384/2765] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:36.605990 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2d86-602d-4769-99c0-b53d17acd253 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:36.667567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2d86-602d-4769-99c0-b53d17acd253 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:36.672036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:36.672682 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:36.672964 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:36.673232 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] PATH_INFO: `/v3/users/2d04e5ce68cf48af9c5d5c73c5574181` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:36.674640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d04e5ce68cf48af9c5d5c73c5574181)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:36.676627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:36.725047 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d04e5ce68cf48af9c5d5c73c5574181'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:36.731936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d04e5ce68cf48af9c5d5c73c5574181'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:36.732627 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d04e5ce68cf48af9c5d5c73c5574181'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:36.745113 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d04e5ce68cf48af9c5d5c73c5574181'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:36.754360 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d04e5ce68cf48af9c5d5c73c5574181'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:36.760343 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1382/2766] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:34:36 2022] DELETE /identity/v3/users/2d04e5ce68cf48af9c5d5c73c5574181 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:36.764765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b489c5-5eba-42e8-a4f4-3caea25a4687 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:36.765245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b489c5-5eba-42e8-a4f4-3caea25a4687 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:36.765364 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b489c5-5eba-42e8-a4f4-3caea25a4687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:36.784713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-92b489c5-5eba-42e8-a4f4-3caea25a4687 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:36.785932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-92b489c5-5eba-42e8-a4f4-3caea25a4687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:36.847030 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1385/2767] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:34:36.857518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:36.858982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:36.893579 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:36.922568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc3190-a922-4d9a-b4f6-68b970738487 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:36.927051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16752ed9-2178-4a4a-8503-bca3d55d64e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:36.927644 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16752ed9-2178-4a4a-8503-bca3d55d64e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:36.927920 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16752ed9-2178-4a4a-8503-bca3d55d64e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:36.928189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16752ed9-2178-4a4a-8503-bca3d55d64e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:36.937190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16752ed9-2178-4a4a-8503-bca3d55d64e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:36.938626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16752ed9-2178-4a4a-8503-bca3d55d64e6 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:36.948080 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1383/2768] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:37.218144 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b489c5-5eba-42e8-a4f4-3caea25a4687 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:37.219587 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-92b489c5-5eba-42e8-a4f4-3caea25a4687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:37.233221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b489c5-5eba-42e8-a4f4-3caea25a4687 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:37.238674 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:37.239314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:37.239596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:37.239872 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] PATH_INFO: `/v3/projects/532218762bb84a10ae7bf94bde9147f9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:37.241152 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=532218762bb84a10ae7bf94bde9147f9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:37.242967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:37.284468 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:37.291289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532218762bb84a10ae7bf94bde9147f9', 'request_id': 'req-beaedf17-1c4a-460b-894c-eb9c11d203fc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.296988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532218762bb84a10ae7bf94bde9147f9', 'request_id': 'req-beaedf17-1c4a-460b-894c-eb9c11d203fc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.305271 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532218762bb84a10ae7bf94bde9147f9', 'request_id': 'req-beaedf17-1c4a-460b-894c-eb9c11d203fc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.316722 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] The token cache is being invalidate because project 532218762bb84a10ae7bf94bde9147f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:34:37.317009 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.317628 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.319101 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1386/2769] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:34:37 2022] DELETE /identity/v3/projects/532218762bb84a10ae7bf94bde9147f9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:37.323180 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16752ed9-2178-4a4a-8503-bca3d55d64e6 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:37.333348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16752ed9-2178-4a4a-8503-bca3d55d64e6 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:37.389724 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:37.390255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:37.390531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:37.390798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] PATH_INFO: `/v3/users/bfd67c175eb84514b36033f652b60542` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:37.391890 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd67c175eb84514b36033f652b60542)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:37.394408 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:37.441035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd67c175eb84514b36033f652b60542'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.447671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd67c175eb84514b36033f652b60542'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.448336 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd67c175eb84514b36033f652b60542'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.460910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd67c175eb84514b36033f652b60542'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.468613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cbe323f5-1103-4a71-a727-bbc197965c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd67c175eb84514b36033f652b60542'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.474399 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1384/2770] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:34:37 2022] DELETE /identity/v3/users/bfd67c175eb84514b36033f652b60542 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:37.750602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:37.752610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:37.766993 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaedf17-1c4a-460b-894c-eb9c11d203fc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:37.824044 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:37.824713 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:37.824998 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:37.825275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] PATH_INFO: `/v3/projects/481ad3299f024c1f9237c682bfb45ba2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:37.826401 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=481ad3299f024c1f9237c682bfb45ba2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:37.828335 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:37.876518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481ad3299f024c1f9237c682bfb45ba2', 'request_id': 'req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.882350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481ad3299f024c1f9237c682bfb45ba2', 'request_id': 'req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.889500 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481ad3299f024c1f9237c682bfb45ba2', 'request_id': 'req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.901905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] The token cache is being invalidate because project 481ad3299f024c1f9237c682bfb45ba2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:34:37.902106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.902736 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91b84a21-1b8b-45f4-8342-e33bc9f9ef36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:37.904204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1387/2771] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:34:37 2022] DELETE /identity/v3/projects/481ad3299f024c1f9237c682bfb45ba2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:37.909090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1cb17-4892-403e-aa0a-d659692668d4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:37.909572 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1cb17-4892-403e-aa0a-d659692668d4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:37.909690 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1cb17-4892-403e-aa0a-d659692668d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:37.928771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-daa1cb17-4892-403e-aa0a-d659692668d4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:37.930237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-daa1cb17-4892-403e-aa0a-d659692668d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:37.973915 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:37.990040 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1385/2772] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:37.997845 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1618f-aac3-494e-b721-f073d73ef6bc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:37.998132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1618f-aac3-494e-b721-f073d73ef6bc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:37.998409 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1618f-aac3-494e-b721-f073d73ef6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:38.018290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-51b1618f-aac3-494e-b721-f073d73ef6bc None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:38.020522 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-51b1618f-aac3-494e-b721-f073d73ef6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:38.028621 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1388/2773] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:38.032831 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1cb17-4892-403e-aa0a-d659692668d4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:38.042518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1cb17-4892-403e-aa0a-d659692668d4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:38.047060 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b357b-7f60-425a-996e-976256f50d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:38.047662 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b357b-7f60-425a-996e-976256f50d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:38.047945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b357b-7f60-425a-996e-976256f50d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:38.048219 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b357b-7f60-425a-996e-976256f50d28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:38.049404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b357b-7f60-425a-996e-976256f50d28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:34:38.049713 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b357b-7f60-425a-996e-976256f50d28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:38.051321 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b357b-7f60-425a-996e-976256f50d28 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:38.063221 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1386/2774] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 15:34:38.068289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1618f-aac3-494e-b721-f073d73ef6bc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:38.077569 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1618f-aac3-494e-b721-f073d73ef6bc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:38.082279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d1535-a12e-4b9a-8168-206b4e3fa15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:38.082891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d1535-a12e-4b9a-8168-206b4e3fa15f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:38.083170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d1535-a12e-4b9a-8168-206b4e3fa15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:38.083442 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d1535-a12e-4b9a-8168-206b4e3fa15f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:38.084056 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d1535-a12e-4b9a-8168-206b4e3fa15f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:38.086002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d1535-a12e-4b9a-8168-206b4e3fa15f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:38.112781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1389/2775] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:34:38 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:38.116787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b357b-7f60-425a-996e-976256f50d28 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:38.126472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b357b-7f60-425a-996e-976256f50d28 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:38.184598 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dcfa5-994a-4613-aaf2-cea0d6bb4df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:38.185115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcfa5-994a-4613-aaf2-cea0d6bb4df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:38.185392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcfa5-994a-4613-aaf2-cea0d6bb4df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:38.185663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcfa5-994a-4613-aaf2-cea0d6bb4df1 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:38.186292 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07dcfa5-994a-4613-aaf2-cea0d6bb4df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:38.188205 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07dcfa5-994a-4613-aaf2-cea0d6bb4df1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:38.211051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1387/2776] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:34:38 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:38.215110 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d1535-a12e-4b9a-8168-206b4e3fa15f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:38.227680 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d1535-a12e-4b9a-8168-206b4e3fa15f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:38.232672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c902d1b-71f4-4f7c-b97a-3ed821b2162b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:38.233266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c902d1b-71f4-4f7c-b97a-3ed821b2162b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:38.233540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c902d1b-71f4-4f7c-b97a-3ed821b2162b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:38.233807 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c902d1b-71f4-4f7c-b97a-3ed821b2162b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:38.234442 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c902d1b-71f4-4f7c-b97a-3ed821b2162b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:38.236043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c902d1b-71f4-4f7c-b97a-3ed821b2162b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:38.247032 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1390/2777] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:38.251275 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dcfa5-994a-4613-aaf2-cea0d6bb4df1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:38.262000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dcfa5-994a-4613-aaf2-cea0d6bb4df1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:38.266636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56688749-84e4-4dc6-9199-bb9d2e707152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:38.267223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56688749-84e4-4dc6-9199-bb9d2e707152 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:38.267525 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56688749-84e4-4dc6-9199-bb9d2e707152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:38.267803 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56688749-84e4-4dc6-9199-bb9d2e707152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:38.268473 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56688749-84e4-4dc6-9199-bb9d2e707152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:38.270100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56688749-84e4-4dc6-9199-bb9d2e707152 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:38.280374 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1388/2778] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:38.284627 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c902d1b-71f4-4f7c-b97a-3ed821b2162b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:38.294602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c902d1b-71f4-4f7c-b97a-3ed821b2162b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:38.300046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079faa3-69a1-4e10-8192-0dfbe7a727b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:38.300737 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079faa3-69a1-4e10-8192-0dfbe7a727b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:38.301017 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079faa3-69a1-4e10-8192-0dfbe7a727b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:38.301291 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079faa3-69a1-4e10-8192-0dfbe7a727b5 admin admin] PATH_INFO: `/v3/projects/40cef7bfcd3047449cfe151f890339fc/users/37dde8ec2253461780e5328ace10f5e5/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:38.317878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079faa3-69a1-4e10-8192-0dfbe7a727b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cef7bfcd3047449cfe151f890339fc, user_id=37dde8ec2253461780e5328ace10f5e5, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:38.320602 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079faa3-69a1-4e10-8192-0dfbe7a727b5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:38.323538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:38.335749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1391/2779] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:34:38 2022] PUT /identity/v3/projects/40cef7bfcd3047449cfe151f890339fc/users/37dde8ec2253461780e5328ace10f5e5/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:38.340858 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141626f-ef09-49d9-a549-ab8306c2f2ef None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:38.341142 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141626f-ef09-49d9-a549-ab8306c2f2ef None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:38.341416 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141626f-ef09-49d9-a549-ab8306c2f2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:38.360735 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6141626f-ef09-49d9-a549-ab8306c2f2ef None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:38.362533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6141626f-ef09-49d9-a549-ab8306c2f2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:38.418972 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1389/2780] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:34:38.429594 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079faa3-69a1-4e10-8192-0dfbe7a727b5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:38.431286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b079faa3-69a1-4e10-8192-0dfbe7a727b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:38.489860 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079faa3-69a1-4e10-8192-0dfbe7a727b5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:38.493488 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1a6dd-6ca0-402f-a8e0-c0e43cf35faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:38.494096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1a6dd-6ca0-402f-a8e0-c0e43cf35faf service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:38.494380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1a6dd-6ca0-402f-a8e0-c0e43cf35faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:38.494651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1a6dd-6ca0-402f-a8e0-c0e43cf35faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:38.503672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1a6dd-6ca0-402f-a8e0-c0e43cf35faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:38.505149 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd1a6dd-6ca0-402f-a8e0-c0e43cf35faf service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:38.514447 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1392/2781] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:39.115385 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a17a3b-6566-4a96-a81c-38a0bcfe0579 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:39.115945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a17a3b-6566-4a96-a81c-38a0bcfe0579 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:39.115945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a17a3b-6566-4a96-a81c-38a0bcfe0579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:39.144901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-29a17a3b-6566-4a96-a81c-38a0bcfe0579 None None] MFA Rules not processed for user `37dde8ec2253461780e5328ace10f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:39.146150 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-29a17a3b-6566-4a96-a81c-38a0bcfe0579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:39.191809 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:39.217476 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1390/2782] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:39.222932 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1a6dd-6ca0-402f-a8e0-c0e43cf35faf service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:39.233657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1a6dd-6ca0-402f-a8e0-c0e43cf35faf service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:39.238825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5f2a8-377e-4824-826f-5a05837d3482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:39.239418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5f2a8-377e-4824-826f-5a05837d3482 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:39.239703 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5f2a8-377e-4824-826f-5a05837d3482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:39.239976 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5f2a8-377e-4824-826f-5a05837d3482 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:39.240633 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5f2a8-377e-4824-826f-5a05837d3482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:39.242545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5f2a8-377e-4824-826f-5a05837d3482 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:39.269660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1393/2783] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:34:39 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:34:39.273638 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a17a3b-6566-4a96-a81c-38a0bcfe0579 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:39.285306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a17a3b-6566-4a96-a81c-38a0bcfe0579 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:39.340781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7fa1b-c9a9-4409-b0a8-782dcfbfb24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:39.341407 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7fa1b-c9a9-4409-b0a8-782dcfbfb24b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:39.341683 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7fa1b-c9a9-4409-b0a8-782dcfbfb24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:39.341953 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7fa1b-c9a9-4409-b0a8-782dcfbfb24b admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:39.342574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7fa1b-c9a9-4409-b0a8-782dcfbfb24b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:39.344530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7fa1b-c9a9-4409-b0a8-782dcfbfb24b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:39.367212 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1391/2784] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:34:39 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:34:39.371183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5f2a8-377e-4824-826f-5a05837d3482 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:39.381942 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5f2a8-377e-4824-826f-5a05837d3482 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:39.392595 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25c50c-822f-4c48-abff-0d0a835b628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:39.393265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25c50c-822f-4c48-abff-0d0a835b628b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:39.393539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25c50c-822f-4c48-abff-0d0a835b628b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:39.393833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25c50c-822f-4c48-abff-0d0a835b628b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:39.394460 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25c50c-822f-4c48-abff-0d0a835b628b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:39.396079 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25c50c-822f-4c48-abff-0d0a835b628b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:39.406904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1394/2785] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:39.411994 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7fa1b-c9a9-4409-b0a8-782dcfbfb24b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:39.422062 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7fa1b-c9a9-4409-b0a8-782dcfbfb24b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:39.427377 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f87a7-e2f8-4fd6-9160-7d2b7ab0bcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:39.428021 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f87a7-e2f8-4fd6-9160-7d2b7ab0bcc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:39.428300 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f87a7-e2f8-4fd6-9160-7d2b7ab0bcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:39.428606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f87a7-e2f8-4fd6-9160-7d2b7ab0bcc3 admin admin] PATH_INFO: `/v3/projects/c6a682667b4e4a619fff236870c20e23/users/209cbc71244b4e878b1ef91728ba52bd/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:39.443834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f87a7-e2f8-4fd6-9160-7d2b7ab0bcc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a682667b4e4a619fff236870c20e23, user_id=209cbc71244b4e878b1ef91728ba52bd, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:39.446623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f87a7-e2f8-4fd6-9160-7d2b7ab0bcc3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:39.458849 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1392/2786] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:34:39 2022] PUT /identity/v3/projects/c6a682667b4e4a619fff236870c20e23/users/209cbc71244b4e878b1ef91728ba52bd/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:39.462717 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25c50c-822f-4c48-abff-0d0a835b628b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:39.473420 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25c50c-822f-4c48-abff-0d0a835b628b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:39.528810 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8370fb-fff7-4ff1-8f4c-e0c185145eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:39.529670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8370fb-fff7-4ff1-8f4c-e0c185145eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:39.530024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8370fb-fff7-4ff1-8f4c-e0c185145eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:39.530301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8370fb-fff7-4ff1-8f4c-e0c185145eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:39.530938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8370fb-fff7-4ff1-8f4c-e0c185145eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:39.532578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8370fb-fff7-4ff1-8f4c-e0c185145eb5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:39.543245 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1395/2787] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:39.547453 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f87a7-e2f8-4fd6-9160-7d2b7ab0bcc3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:39.558632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f87a7-e2f8-4fd6-9160-7d2b7ab0bcc3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:39.563189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066216e5-2fac-4023-8b1d-9f57ffd2dc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:39.563826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066216e5-2fac-4023-8b1d-9f57ffd2dc08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:39.564099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066216e5-2fac-4023-8b1d-9f57ffd2dc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:39.565100 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066216e5-2fac-4023-8b1d-9f57ffd2dc08 admin admin] PATH_INFO: `/v3/projects/c6a682667b4e4a619fff236870c20e23/users/209cbc71244b4e878b1ef91728ba52bd/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:39.567345 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066216e5-2fac-4023-8b1d-9f57ffd2dc08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a682667b4e4a619fff236870c20e23, user_id=209cbc71244b4e878b1ef91728ba52bd, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:39.570017 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066216e5-2fac-4023-8b1d-9f57ffd2dc08 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:39.579868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1393/2788] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:34:39 2022] PUT /identity/v3/projects/c6a682667b4e4a619fff236870c20e23/users/209cbc71244b4e878b1ef91728ba52bd/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:34:40.179456 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fcec4-47ad-4843-b21b-54fb64a492a3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:40.179829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fcec4-47ad-4843-b21b-54fb64a492a3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:40.180208 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fcec4-47ad-4843-b21b-54fb64a492a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:40.209466 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-118fcec4-47ad-4843-b21b-54fb64a492a3 None None] MFA Rules not processed for user `209cbc71244b4e878b1ef91728ba52bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:40.210904 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-118fcec4-47ad-4843-b21b-54fb64a492a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:40.266399 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:40.290235 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1396/2789] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:40.301975 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066216e5-2fac-4023-8b1d-9f57ffd2dc08 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:40.303708 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-066216e5-2fac-4023-8b1d-9f57ffd2dc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:40.364656 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066216e5-2fac-4023-8b1d-9f57ffd2dc08 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:40.368173 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff8e74-3c01-4d9b-a9b5-c629847f896b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:40.368810 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff8e74-3c01-4d9b-a9b5-c629847f896b service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:40.369090 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff8e74-3c01-4d9b-a9b5-c629847f896b service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:40.369363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff8e74-3c01-4d9b-a9b5-c629847f896b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:40.416106 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:40.426919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff8e74-3c01-4d9b-a9b5-c629847f896b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:40.428441 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff8e74-3c01-4d9b-a9b5-c629847f896b service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:40.437745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1394/2790] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:41.791096 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fcec4-47ad-4843-b21b-54fb64a492a3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:41.792545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-118fcec4-47ad-4843-b21b-54fb64a492a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:41.805063 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fcec4-47ad-4843-b21b-54fb64a492a3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:41.808644 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e751a3-9475-4e47-ba66-83f6cd38076c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:41.809236 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e751a3-9475-4e47-ba66-83f6cd38076c service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:41.809514 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e751a3-9475-4e47-ba66-83f6cd38076c service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:41.809785 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e751a3-9475-4e47-ba66-83f6cd38076c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:41.819085 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e751a3-9475-4e47-ba66-83f6cd38076c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:41.820585 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e751a3-9475-4e47-ba66-83f6cd38076c service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:41.829885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1397/2791] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:50.433824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:51.825709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=12 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:54.239353 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff8e74-3c01-4d9b-a9b5-c629847f896b service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:54.241484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dbff8e74-3c01-4d9b-a9b5-c629847f896b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:54.315129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff8e74-3c01-4d9b-a9b5-c629847f896b service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:54.320824 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064c55c-dfbc-46fc-8046-3d6a580e51a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:54.321438 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c55c-dfbc-46fc-8046-3d6a580e51a6 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:54.321715 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c55c-dfbc-46fc-8046-3d6a580e51a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:54.321982 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c55c-dfbc-46fc-8046-3d6a580e51a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:54.323755 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7064c55c-dfbc-46fc-8046-3d6a580e51a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:54.336137 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064c55c-dfbc-46fc-8046-3d6a580e51a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:54.337821 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064c55c-dfbc-46fc-8046-3d6a580e51a6 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:54.348845 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1395/2792] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:54.756700 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e751a3-9475-4e47-ba66-83f6cd38076c service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:54.767867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e751a3-9475-4e47-ba66-83f6cd38076c service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:54.773435 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a1a1b-3fce-4e34-be8c-4b38fae4ab31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:54.774026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a1a1b-3fce-4e34-be8c-4b38fae4ab31 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:54.774308 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a1a1b-3fce-4e34-be8c-4b38fae4ab31 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:54.774573 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a1a1b-3fce-4e34-be8c-4b38fae4ab31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:54.776270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b98a1a1b-3fce-4e34-be8c-4b38fae4ab31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:54.788199 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a1a1b-3fce-4e34-be8c-4b38fae4ab31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:54.789862 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a1a1b-3fce-4e34-be8c-4b38fae4ab31 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:54.801786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1398/2793] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:34:54.819442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064c55c-dfbc-46fc-8046-3d6a580e51a6 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:54.888179 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064c55c-dfbc-46fc-8046-3d6a580e51a6 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:54.892519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:54.893127 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:54.893402 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:54.893670 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] PATH_INFO: `/v3/users/37dde8ec2253461780e5328ace10f5e5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:54.894762 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] RBAC: Authorizing `identity:delete_user(user_id=37dde8ec2253461780e5328ace10f5e5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:54.896852 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:54.942812 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37dde8ec2253461780e5328ace10f5e5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:54.949954 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37dde8ec2253461780e5328ace10f5e5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:54.950636 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37dde8ec2253461780e5328ace10f5e5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:54.962284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37dde8ec2253461780e5328ace10f5e5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:54.969433 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37dde8ec2253461780e5328ace10f5e5'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:54.975425 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1396/2794] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:34:54 2022] DELETE /identity/v3/users/37dde8ec2253461780e5328ace10f5e5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:55.377228 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a1a1b-3fce-4e34-be8c-4b38fae4ab31 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:55.388607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a1a1b-3fce-4e34-be8c-4b38fae4ab31 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:55.446189 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:55.446905 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:55.447192 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:55.447472 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] PATH_INFO: `/v3/projects/40cef7bfcd3047449cfe151f890339fc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:55.448654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cef7bfcd3047449cfe151f890339fc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:55.450574 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:55.495938 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cef7bfcd3047449cfe151f890339fc', 'request_id': 'req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.501424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cef7bfcd3047449cfe151f890339fc', 'request_id': 'req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.513385 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cef7bfcd3047449cfe151f890339fc', 'request_id': 'req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.524953 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] The token cache is being invalidate because project 40cef7bfcd3047449cfe151f890339fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:34:55.525289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.526190 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.527802 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1399/2795] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:34:55 2022] DELETE /identity/v3/projects/40cef7bfcd3047449cfe151f890339fc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:55.531808 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:55.533286 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:55.547074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c422596-db19-40e1-ba72-0d207456c56a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:55.552531 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:55.607000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:55.607624 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:55.607910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:55.608180 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] PATH_INFO: `/v3/users/209cbc71244b4e878b1ef91728ba52bd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:55.609342 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] RBAC: Authorizing `identity:delete_user(user_id=209cbc71244b4e878b1ef91728ba52bd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:55.611259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:55.659339 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '209cbc71244b4e878b1ef91728ba52bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.667251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '209cbc71244b4e878b1ef91728ba52bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.667944 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '209cbc71244b4e878b1ef91728ba52bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.680480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '209cbc71244b4e878b1ef91728ba52bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.688007 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d212721a-1c53-48d3-9a4a-04619b26978f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '209cbc71244b4e878b1ef91728ba52bd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:55.695140 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1397/2796] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:34:55 2022] DELETE /identity/v3/users/209cbc71244b4e878b1ef91728ba52bd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:55.871754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:55.873316 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:55.889141 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2929e-fad4-47d9-b3b9-b8f8b9d2b62d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:55.948452 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:55.948997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:55.949279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:55.949551 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] PATH_INFO: `/v3/projects/c6a682667b4e4a619fff236870c20e23` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:55.950659 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a682667b4e4a619fff236870c20e23)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:55.952619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:55.973561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:55.998278 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a682667b4e4a619fff236870c20e23', 'request_id': 'req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:56.003672 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a682667b4e4a619fff236870c20e23', 'request_id': 'req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:56.011426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a682667b4e4a619fff236870c20e23', 'request_id': 'req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:56.023465 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] The token cache is being invalidate because project c6a682667b4e4a619fff236870c20e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:34:56.023776 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:56.024434 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf86394e-6360-4a7a-b9e7-e3d22e48e3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:34:56.025906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1400/2797] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:34:55 2022] DELETE /identity/v3/projects/c6a682667b4e4a619fff236870c20e23 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:56.030757 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2aba8-d24a-42ce-85d9-019a67324c56 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.031028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2aba8-d24a-42ce-85d9-019a67324c56 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.031306 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2aba8-d24a-42ce-85d9-019a67324c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.050671 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0ec2aba8-d24a-42ce-85d9-019a67324c56 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:56.051900 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0ec2aba8-d24a-42ce-85d9-019a67324c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:56.110198 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1398/2798] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:56.119260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d4988-c860-42a7-a473-5c797da41bbe None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.119544 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d4988-c860-42a7-a473-5c797da41bbe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.119819 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d4988-c860-42a7-a473-5c797da41bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.141000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e88d4988-c860-42a7-a473-5c797da41bbe None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:56.142490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e88d4988-c860-42a7-a473-5c797da41bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:56.152136 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1401/2799] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:56.156545 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2aba8-d24a-42ce-85d9-019a67324c56 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.166732 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2aba8-d24a-42ce-85d9-019a67324c56 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.171231 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120feba5-cc2e-4f1d-9e4c-17f5a71d0d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.171911 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120feba5-cc2e-4f1d-9e4c-17f5a71d0d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.172207 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120feba5-cc2e-4f1d-9e4c-17f5a71d0d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.172503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120feba5-cc2e-4f1d-9e4c-17f5a71d0d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.173141 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120feba5-cc2e-4f1d-9e4c-17f5a71d0d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:34:56.173439 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120feba5-cc2e-4f1d-9e4c-17f5a71d0d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.175033 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120feba5-cc2e-4f1d-9e4c-17f5a71d0d63 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.186862 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1399/2800] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:34:56.191024 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d4988-c860-42a7-a473-5c797da41bbe None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.201256 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d4988-c860-42a7-a473-5c797da41bbe None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.205767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c30977-7136-4606-b2de-398b4eb8d98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.206347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c30977-7136-4606-b2de-398b4eb8d98a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.206622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c30977-7136-4606-b2de-398b4eb8d98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.206891 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c30977-7136-4606-b2de-398b4eb8d98a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.208064 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c30977-7136-4606-b2de-398b4eb8d98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.210464 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c30977-7136-4606-b2de-398b4eb8d98a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.237821 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1402/2801] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:34:56 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:56.241942 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120feba5-cc2e-4f1d-9e4c-17f5a71d0d63 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.252025 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120feba5-cc2e-4f1d-9e4c-17f5a71d0d63 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.263997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:56.311199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6eae0f-b445-41b8-a4a8-64de0dad700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.311780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6eae0f-b445-41b8-a4a8-64de0dad700d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.312057 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6eae0f-b445-41b8-a4a8-64de0dad700d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.312325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6eae0f-b445-41b8-a4a8-64de0dad700d admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.312971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6eae0f-b445-41b8-a4a8-64de0dad700d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.314875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6eae0f-b445-41b8-a4a8-64de0dad700d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.337495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1400/2802] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:34:56 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:56.341747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c30977-7136-4606-b2de-398b4eb8d98a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.353207 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c30977-7136-4606-b2de-398b4eb8d98a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.358039 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37b963-c656-4698-9c26-fb955258f564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.358668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37b963-c656-4698-9c26-fb955258f564 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.358946 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37b963-c656-4698-9c26-fb955258f564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.359220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37b963-c656-4698-9c26-fb955258f564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.359867 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37b963-c656-4698-9c26-fb955258f564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.361513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37b963-c656-4698-9c26-fb955258f564 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.371716 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1403/2803] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:56.375622 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6eae0f-b445-41b8-a4a8-64de0dad700d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.386056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6eae0f-b445-41b8-a4a8-64de0dad700d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.390517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde66f27-47b5-4ab1-90b4-9fad6089ea0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.391096 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde66f27-47b5-4ab1-90b4-9fad6089ea0f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.391372 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde66f27-47b5-4ab1-90b4-9fad6089ea0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.391643 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde66f27-47b5-4ab1-90b4-9fad6089ea0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.392274 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde66f27-47b5-4ab1-90b4-9fad6089ea0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.393898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde66f27-47b5-4ab1-90b4-9fad6089ea0f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.404993 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1401/2804] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:34:56.409021 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37b963-c656-4698-9c26-fb955258f564 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.419664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37b963-c656-4698-9c26-fb955258f564 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.424260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f4886-b63a-43d3-abe5-212af56b0401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.424937 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f4886-b63a-43d3-abe5-212af56b0401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.425216 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f4886-b63a-43d3-abe5-212af56b0401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.425487 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f4886-b63a-43d3-abe5-212af56b0401 admin admin] PATH_INFO: `/v3/projects/1d24d83ad2a84e4d8265aaac89ca41c5/users/aa4399bc27af4ff6a57a7d0789910f98/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.441252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f4886-b63a-43d3-abe5-212af56b0401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d24d83ad2a84e4d8265aaac89ca41c5, user_id=aa4399bc27af4ff6a57a7d0789910f98, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.443994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f4886-b63a-43d3-abe5-212af56b0401 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.457097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1404/2805] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:34:56 2022] PUT /identity/v3/projects/1d24d83ad2a84e4d8265aaac89ca41c5/users/aa4399bc27af4ff6a57a7d0789910f98/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:56.519284 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688993c1-0433-4bc6-abcf-5e022103a509 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.519568 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688993c1-0433-4bc6-abcf-5e022103a509 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.519846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688993c1-0433-4bc6-abcf-5e022103a509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.548685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-688993c1-0433-4bc6-abcf-5e022103a509 None None] MFA Rules not processed for user `aa4399bc27af4ff6a57a7d0789910f98`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:56.549893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-688993c1-0433-4bc6-abcf-5e022103a509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:56.615369 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1402/2806] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:56.619954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f4886-b63a-43d3-abe5-212af56b0401 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.687246 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f4886-b63a-43d3-abe5-212af56b0401 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.692897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ace20-202c-4dbd-9c26-29ce4a739291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.693503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ace20-202c-4dbd-9c26-29ce4a739291 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.693781 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ace20-202c-4dbd-9c26-29ce4a739291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.694049 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ace20-202c-4dbd-9c26-29ce4a739291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.694670 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ace20-202c-4dbd-9c26-29ce4a739291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.696617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347ace20-202c-4dbd-9c26-29ce4a739291 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.722323 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1405/2807] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:34:56 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:56.727223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688993c1-0433-4bc6-abcf-5e022103a509 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.737797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688993c1-0433-4bc6-abcf-5e022103a509 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.793905 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf8384-5b41-4103-a9e2-87c894d597ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.794494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf8384-5b41-4103-a9e2-87c894d597ef admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.794771 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf8384-5b41-4103-a9e2-87c894d597ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.795039 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf8384-5b41-4103-a9e2-87c894d597ef admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.795668 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf8384-5b41-4103-a9e2-87c894d597ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.797611 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf8384-5b41-4103-a9e2-87c894d597ef admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.820003 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1403/2808] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:34:56 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:34:56.824071 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ace20-202c-4dbd-9c26-29ce4a739291 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.835043 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ace20-202c-4dbd-9c26-29ce4a739291 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.839654 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fecf7-8e62-4721-abac-ba7f9cdc6de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.840275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fecf7-8e62-4721-abac-ba7f9cdc6de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.840575 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fecf7-8e62-4721-abac-ba7f9cdc6de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.840851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fecf7-8e62-4721-abac-ba7f9cdc6de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.841491 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fecf7-8e62-4721-abac-ba7f9cdc6de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.843098 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728fecf7-8e62-4721-abac-ba7f9cdc6de4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.854600 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1406/2809] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:56.858010 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf8384-5b41-4103-a9e2-87c894d597ef admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.868220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf8384-5b41-4103-a9e2-87c894d597ef admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.872882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3027379-1fdd-487e-97a4-2e1fd15f506b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.873519 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3027379-1fdd-487e-97a4-2e1fd15f506b admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.873826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3027379-1fdd-487e-97a4-2e1fd15f506b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.874097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3027379-1fdd-487e-97a4-2e1fd15f506b admin admin] PATH_INFO: `/v3/projects/0b81d89ac5ac43bd8c8230a76648991a/users/eca4a4d1a94d4bd38f49c072f4bcb5ec/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.885850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:56.890103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3027379-1fdd-487e-97a4-2e1fd15f506b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b81d89ac5ac43bd8c8230a76648991a, user_id=eca4a4d1a94d4bd38f49c072f4bcb5ec, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.893226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3027379-1fdd-487e-97a4-2e1fd15f506b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.906278 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1404/2810] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:34:56 2022] PUT /identity/v3/projects/0b81d89ac5ac43bd8c8230a76648991a/users/eca4a4d1a94d4bd38f49c072f4bcb5ec/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:56.910221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fecf7-8e62-4721-abac-ba7f9cdc6de4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:56.920640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fecf7-8e62-4721-abac-ba7f9cdc6de4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:56.933313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:34:56.982026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f77f1-83ab-4fbd-8d84-d8f30b140d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:56.982538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f77f1-83ab-4fbd-8d84-d8f30b140d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:56.982817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f77f1-83ab-4fbd-8d84-d8f30b140d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:56.983087 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f77f1-83ab-4fbd-8d84-d8f30b140d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:56.983719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f77f1-83ab-4fbd-8d84-d8f30b140d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:56.985358 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f77f1-83ab-4fbd-8d84-d8f30b140d60 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:56.995172 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1407/2811] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:34:56.999375 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3027379-1fdd-487e-97a4-2e1fd15f506b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:57.010851 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3027379-1fdd-487e-97a4-2e1fd15f506b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:57.015532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04186b-3e5f-428b-86fd-6a40d8a054c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:57.016186 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04186b-3e5f-428b-86fd-6a40d8a054c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:57.016508 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04186b-3e5f-428b-86fd-6a40d8a054c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:57.016788 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04186b-3e5f-428b-86fd-6a40d8a054c9 admin admin] PATH_INFO: `/v3/projects/0b81d89ac5ac43bd8c8230a76648991a/users/eca4a4d1a94d4bd38f49c072f4bcb5ec/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:57.018686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04186b-3e5f-428b-86fd-6a40d8a054c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b81d89ac5ac43bd8c8230a76648991a, user_id=eca4a4d1a94d4bd38f49c072f4bcb5ec, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:57.022001 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04186b-3e5f-428b-86fd-6a40d8a054c9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:57.032109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1405/2812] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:34:56 2022] PUT /identity/v3/projects/0b81d89ac5ac43bd8c8230a76648991a/users/eca4a4d1a94d4bd38f49c072f4bcb5ec/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:34:57.042028 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9bb2-8938-4c2f-95c6-588cb5cd1bf6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:57.042313 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9bb2-8938-4c2f-95c6-588cb5cd1bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:57.042622 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9bb2-8938-4c2f-95c6-588cb5cd1bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:57.071503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a36c9bb2-8938-4c2f-95c6-588cb5cd1bf6 None None] MFA Rules not processed for user `eca4a4d1a94d4bd38f49c072f4bcb5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:34:57.072743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a36c9bb2-8938-4c2f-95c6-588cb5cd1bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:57.141657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1408/2813] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:34:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:34:57.153056 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04186b-3e5f-428b-86fd-6a40d8a054c9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:34:57.154745 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cf04186b-3e5f-428b-86fd-6a40d8a054c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:34:57.211837 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04186b-3e5f-428b-86fd-6a40d8a054c9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:34:57.215518 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177b422-3bbc-4970-93b3-95555ccf06d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:34:57.216111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b422-3bbc-4970-93b3-95555ccf06d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:34:57.216410 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b422-3bbc-4970-93b3-95555ccf06d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:34:57.216686 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177b422-3bbc-4970-93b3-95555ccf06d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:34:57.272620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177b422-3bbc-4970-93b3-95555ccf06d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:34:57.274050 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177b422-3bbc-4970-93b3-95555ccf06d8 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:34:57.282613 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1406/2814] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:34:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:01.474807 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9bb2-8938-4c2f-95c6-588cb5cd1bf6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:01.476303 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a36c9bb2-8938-4c2f-95c6-588cb5cd1bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:01.488550 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c9bb2-8938-4c2f-95c6-588cb5cd1bf6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:01.492170 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f9ced-fc8c-4ef0-98f4-d4e630ad859a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:01.492800 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f9ced-fc8c-4ef0-98f4-d4e630ad859a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:01.493080 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f9ced-fc8c-4ef0-98f4-d4e630ad859a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:01.493351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f9ced-fc8c-4ef0-98f4-d4e630ad859a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:01.502167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f9ced-fc8c-4ef0-98f4-d4e630ad859a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:01.503607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f9ced-fc8c-4ef0-98f4-d4e630ad859a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:01.513571 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1409/2815] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:06.405401 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177b422-3bbc-4970-93b3-95555ccf06d8 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:06.466759 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177b422-3bbc-4970-93b3-95555ccf06d8 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:06.471213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:06.471834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:06.472114 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:06.472404 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] PATH_INFO: `/v3/users/aa4399bc27af4ff6a57a7d0789910f98` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:06.473503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa4399bc27af4ff6a57a7d0789910f98)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:06.475418 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:06.510659 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:06.523279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa4399bc27af4ff6a57a7d0789910f98'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:06.530154 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa4399bc27af4ff6a57a7d0789910f98'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:06.530843 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa4399bc27af4ff6a57a7d0789910f98'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:06.543034 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa4399bc27af4ff6a57a7d0789910f98'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:06.549805 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa4399bc27af4ff6a57a7d0789910f98'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:06.556899 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1407/2816] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:35:06 2022] DELETE /identity/v3/users/aa4399bc27af4ff6a57a7d0789910f98 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:06.561728 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be100a-e9ee-4c37-ab45-313a16b27658 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:06.562260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be100a-e9ee-4c37-ab45-313a16b27658 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:06.562397 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be100a-e9ee-4c37-ab45-313a16b27658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:06.581359 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-83be100a-e9ee-4c37-ab45-313a16b27658 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:06.582548 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-83be100a-e9ee-4c37-ab45-313a16b27658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:06.640893 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1410/2817] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:06.651640 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:06.653099 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:06.711024 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b5add-89df-4a75-9e46-aeb34003aacc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:06.714766 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe1c22-39bc-46c6-823c-3fecff44828c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:06.715379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe1c22-39bc-46c6-823c-3fecff44828c service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:06.715654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe1c22-39bc-46c6-823c-3fecff44828c service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:06.715923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe1c22-39bc-46c6-823c-3fecff44828c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:06.724587 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe1c22-39bc-46c6-823c-3fecff44828c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:06.726012 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe1c22-39bc-46c6-823c-3fecff44828c service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:06.735480 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1408/2818] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 15:35:07.003623 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be100a-e9ee-4c37-ab45-313a16b27658 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:07.005081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-83be100a-e9ee-4c37-ab45-313a16b27658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:07.017668 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be100a-e9ee-4c37-ab45-313a16b27658 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:07.022395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:07.023174 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:07.023424 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:07.023697 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] PATH_INFO: `/v3/projects/1d24d83ad2a84e4d8265aaac89ca41c5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:07.025559 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d24d83ad2a84e4d8265aaac89ca41c5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:07.028426 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:07.052666 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:07.076461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d24d83ad2a84e4d8265aaac89ca41c5', 'request_id': 'req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.082066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d24d83ad2a84e4d8265aaac89ca41c5', 'request_id': 'req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.090873 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d24d83ad2a84e4d8265aaac89ca41c5', 'request_id': 'req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.101901 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] The token cache is being invalidate because project 1d24d83ad2a84e4d8265aaac89ca41c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:35:07.102978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.103694 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.105198 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1411/2819] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:35:07 2022] DELETE /identity/v3/projects/1d24d83ad2a84e4d8265aaac89ca41c5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:07.109279 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe1c22-39bc-46c6-823c-3fecff44828c service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:07.118997 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe1c22-39bc-46c6-823c-3fecff44828c service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:07.173623 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:07.174246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:07.174527 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:07.174800 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] PATH_INFO: `/v3/users/eca4a4d1a94d4bd38f49c072f4bcb5ec` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:07.175910 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca4a4d1a94d4bd38f49c072f4bcb5ec)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:07.177931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:07.222241 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca4a4d1a94d4bd38f49c072f4bcb5ec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.227536 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:07.229733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca4a4d1a94d4bd38f49c072f4bcb5ec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.230602 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca4a4d1a94d4bd38f49c072f4bcb5ec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.243223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca4a4d1a94d4bd38f49c072f4bcb5ec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.250072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-50a1bbdb-6693-426d-95ea-5fff883ec896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca4a4d1a94d4bd38f49c072f4bcb5ec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.255875 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1409/2820] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:35:07 2022] DELETE /identity/v3/users/eca4a4d1a94d4bd38f49c072f4bcb5ec => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:07.523564 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:07.525727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:07.541297 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff872f-abd1-4a3a-9535-b9ca24de9c24 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:07.600095 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:07.600743 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:07.601026 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:07.601301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] PATH_INFO: `/v3/projects/0b81d89ac5ac43bd8c8230a76648991a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:07.602471 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b81d89ac5ac43bd8c8230a76648991a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:07.604455 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:07.651031 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b81d89ac5ac43bd8c8230a76648991a', 'request_id': 'req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.657123 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b81d89ac5ac43bd8c8230a76648991a', 'request_id': 'req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.668360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b81d89ac5ac43bd8c8230a76648991a', 'request_id': 'req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.683919 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] The token cache is being invalidate because project 0b81d89ac5ac43bd8c8230a76648991a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:35:07.684214 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.684925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-04675ba1-a2e9-46b2-8cc8-60a8e8631de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:07.686419 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1412/2821] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:35:07 2022] DELETE /identity/v3/projects/0b81d89ac5ac43bd8c8230a76648991a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:07.692272 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe12111-160c-41c5-ba03-6f64a78bda50 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:07.692781 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe12111-160c-41c5-ba03-6f64a78bda50 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:07.693028 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe12111-160c-41c5-ba03-6f64a78bda50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:07.713526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3fe12111-160c-41c5-ba03-6f64a78bda50 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:07.714651 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3fe12111-160c-41c5-ba03-6f64a78bda50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:07.772382 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1410/2822] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:07.782664 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeba94e-5b0c-4f2f-a499-dd938c5582a4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:07.782950 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeba94e-5b0c-4f2f-a499-dd938c5582a4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:07.783234 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeba94e-5b0c-4f2f-a499-dd938c5582a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:07.804529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aaeba94e-5b0c-4f2f-a499-dd938c5582a4 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:07.805833 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aaeba94e-5b0c-4f2f-a499-dd938c5582a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:07.813784 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1413/2823] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:07.817967 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe12111-160c-41c5-ba03-6f64a78bda50 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:07.827740 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe12111-160c-41c5-ba03-6f64a78bda50 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:07.832341 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6482e0c-fef1-4171-8b36-e35e7bf43b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:07.832962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6482e0c-fef1-4171-8b36-e35e7bf43b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:07.833243 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6482e0c-fef1-4171-8b36-e35e7bf43b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:07.833517 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6482e0c-fef1-4171-8b36-e35e7bf43b53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:07.834163 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6482e0c-fef1-4171-8b36-e35e7bf43b53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:35:07.834467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6482e0c-fef1-4171-8b36-e35e7bf43b53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:07.836102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6482e0c-fef1-4171-8b36-e35e7bf43b53 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:07.848224 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1411/2824] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:35:07.853200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeba94e-5b0c-4f2f-a499-dd938c5582a4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:07.862601 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeba94e-5b0c-4f2f-a499-dd938c5582a4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:07.867310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512260e-d552-4c95-be32-9cc1aa9f80b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:07.868067 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512260e-d552-4c95-be32-9cc1aa9f80b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:07.868360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512260e-d552-4c95-be32-9cc1aa9f80b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:07.868651 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512260e-d552-4c95-be32-9cc1aa9f80b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:07.869304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512260e-d552-4c95-be32-9cc1aa9f80b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:07.871352 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512260e-d552-4c95-be32-9cc1aa9f80b8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:07.901129 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1414/2825] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:35:07 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:35:07.905073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6482e0c-fef1-4171-8b36-e35e7bf43b53 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:07.914917 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6482e0c-fef1-4171-8b36-e35e7bf43b53 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:07.971272 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3195175-fa45-4b7a-8816-e3d66916d66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:07.971794 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3195175-fa45-4b7a-8816-e3d66916d66a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:07.972073 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3195175-fa45-4b7a-8816-e3d66916d66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:07.972344 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3195175-fa45-4b7a-8816-e3d66916d66a admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:07.973002 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3195175-fa45-4b7a-8816-e3d66916d66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:07.974936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3195175-fa45-4b7a-8816-e3d66916d66a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:07.997042 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1412/2826] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:35:07 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:35:08.001411 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512260e-d552-4c95-be32-9cc1aa9f80b8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:08.013408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512260e-d552-4c95-be32-9cc1aa9f80b8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:08.018844 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747c977-4589-48f3-b54f-932d5a180006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:08.019495 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747c977-4589-48f3-b54f-932d5a180006 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:08.019770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747c977-4589-48f3-b54f-932d5a180006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:08.020038 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747c977-4589-48f3-b54f-932d5a180006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:08.020707 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747c977-4589-48f3-b54f-932d5a180006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:08.022330 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747c977-4589-48f3-b54f-932d5a180006 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:08.028360 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:08.034143 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1415/2827] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:35:08.037963 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3195175-fa45-4b7a-8816-e3d66916d66a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:08.048962 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3195175-fa45-4b7a-8816-e3d66916d66a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:08.053697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf7a89-56d5-48cc-9322-191127ab15a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:08.054552 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf7a89-56d5-48cc-9322-191127ab15a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:08.054831 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf7a89-56d5-48cc-9322-191127ab15a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:08.055102 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf7a89-56d5-48cc-9322-191127ab15a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:08.055817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf7a89-56d5-48cc-9322-191127ab15a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:08.057486 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf7a89-56d5-48cc-9322-191127ab15a2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:08.068121 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1413/2828] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:35:08.073107 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747c977-4589-48f3-b54f-932d5a180006 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:08.082931 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747c977-4589-48f3-b54f-932d5a180006 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:08.088324 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c16a6-378e-4f49-a7e7-7b822abd4d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:08.088990 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c16a6-378e-4f49-a7e7-7b822abd4d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:08.089265 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c16a6-378e-4f49-a7e7-7b822abd4d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:08.089534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c16a6-378e-4f49-a7e7-7b822abd4d27 admin admin] PATH_INFO: `/v3/projects/aced88ab250d4681999473410ad84b56/users/dae76d7c88774bc9b44efe8aeb40517a/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:08.106598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c16a6-378e-4f49-a7e7-7b822abd4d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aced88ab250d4681999473410ad84b56, user_id=dae76d7c88774bc9b44efe8aeb40517a, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:08.111604 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c16a6-378e-4f49-a7e7-7b822abd4d27 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:08.124138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1416/2829] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:35:08 2022] PUT /identity/v3/projects/aced88ab250d4681999473410ad84b56/users/dae76d7c88774bc9b44efe8aeb40517a/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:35:08.135752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19368697-4524-4b94-88bf-1913253bce44 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:08.136041 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19368697-4524-4b94-88bf-1913253bce44 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:08.136326 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19368697-4524-4b94-88bf-1913253bce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:08.164532 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-19368697-4524-4b94-88bf-1913253bce44 None None] MFA Rules not processed for user `dae76d7c88774bc9b44efe8aeb40517a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:08.165834 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-19368697-4524-4b94-88bf-1913253bce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:08.191406 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:08.231599 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1414/2830] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:08.243260 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c16a6-378e-4f49-a7e7-7b822abd4d27 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:08.245327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3e4c16a6-378e-4f49-a7e7-7b822abd4d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:08.301168 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c16a6-378e-4f49-a7e7-7b822abd4d27 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:08.305062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b9764-2f53-42b9-a68b-df1d2d8ef343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:08.306275 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b9764-2f53-42b9-a68b-df1d2d8ef343 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:08.306547 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b9764-2f53-42b9-a68b-df1d2d8ef343 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:08.306815 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b9764-2f53-42b9-a68b-df1d2d8ef343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:08.315069 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b9764-2f53-42b9-a68b-df1d2d8ef343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:08.316534 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b9764-2f53-42b9-a68b-df1d2d8ef343 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:08.325967 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1417/2831] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:08.344729 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19368697-4524-4b94-88bf-1913253bce44 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:08.409984 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19368697-4524-4b94-88bf-1913253bce44 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:08.414539 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:08.415603 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:08.415886 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:08.416156 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] PATH_INFO: `/v3/users/dae76d7c88774bc9b44efe8aeb40517a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:08.417314 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae76d7c88774bc9b44efe8aeb40517a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:08.419237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:08.464823 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae76d7c88774bc9b44efe8aeb40517a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:08.471655 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae76d7c88774bc9b44efe8aeb40517a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:08.472330 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae76d7c88774bc9b44efe8aeb40517a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:08.483971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae76d7c88774bc9b44efe8aeb40517a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:08.491185 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae76d7c88774bc9b44efe8aeb40517a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:08.497880 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1415/2832] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:35:08 2022] DELETE /identity/v3/users/dae76d7c88774bc9b44efe8aeb40517a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:08.502351 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33bade-8071-4634-9c9d-82344736841c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:08.502640 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33bade-8071-4634-9c9d-82344736841c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:08.502930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33bade-8071-4634-9c9d-82344736841c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:08.521885 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2d33bade-8071-4634-9c9d-82344736841c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:08.523081 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d33bade-8071-4634-9c9d-82344736841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:08.581504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1418/2833] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:08.591251 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:08.592693 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:08.644478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:08.650894 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe6273-8d59-494e-b761-9d639123b485 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:08.655288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee36e21-10d7-47ec-b7a9-786f59f82bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:08.655901 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee36e21-10d7-47ec-b7a9-786f59f82bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:08.656223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee36e21-10d7-47ec-b7a9-786f59f82bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:08.656685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee36e21-10d7-47ec-b7a9-786f59f82bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:08.666302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee36e21-10d7-47ec-b7a9-786f59f82bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:08.667765 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee36e21-10d7-47ec-b7a9-786f59f82bc2 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:08.678530 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1416/2834] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:08.942719 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33bade-8071-4634-9c9d-82344736841c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:08.944174 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d33bade-8071-4634-9c9d-82344736841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:08.958266 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33bade-8071-4634-9c9d-82344736841c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:08.963727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:08.964408 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:08.964708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:08.965005 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] PATH_INFO: `/v3/projects/aced88ab250d4681999473410ad84b56` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:08.966138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=aced88ab250d4681999473410ad84b56)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:08.968159 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.015279 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aced88ab250d4681999473410ad84b56', 'request_id': 'req-873725db-cdb0-4ef3-ae21-81dbae96efeb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:09.020766 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aced88ab250d4681999473410ad84b56', 'request_id': 'req-873725db-cdb0-4ef3-ae21-81dbae96efeb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:09.028985 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aced88ab250d4681999473410ad84b56', 'request_id': 'req-873725db-cdb0-4ef3-ae21-81dbae96efeb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:09.042771 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] The token cache is being invalidate because project aced88ab250d4681999473410ad84b56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:35:09.043059 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:09.043693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-873725db-cdb0-4ef3-ae21-81dbae96efeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:09.045206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1419/2835] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:35:08 2022] DELETE /identity/v3/projects/aced88ab250d4681999473410ad84b56 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:09.049919 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84213f35-2d24-4430-a956-98fdcafa5466 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.050396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84213f35-2d24-4430-a956-98fdcafa5466 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.050495 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84213f35-2d24-4430-a956-98fdcafa5466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.070220 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-84213f35-2d24-4430-a956-98fdcafa5466 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:09.072192 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-84213f35-2d24-4430-a956-98fdcafa5466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:09.129526 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1417/2836] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:09.137388 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacef86-8c18-4b48-9807-346c78564f1f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.137786 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacef86-8c18-4b48-9807-346c78564f1f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.138130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacef86-8c18-4b48-9807-346c78564f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.146083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:09.159882 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9aacef86-8c18-4b48-9807-346c78564f1f None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:09.161106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9aacef86-8c18-4b48-9807-346c78564f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:09.169637 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1420/2837] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:09.174049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84213f35-2d24-4430-a956-98fdcafa5466 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.184029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84213f35-2d24-4430-a956-98fdcafa5466 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.188461 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add98dcc-57a5-42c9-b910-b7633c28d043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.189046 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add98dcc-57a5-42c9-b910-b7633c28d043 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.189318 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add98dcc-57a5-42c9-b910-b7633c28d043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.189584 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add98dcc-57a5-42c9-b910-b7633c28d043 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.190218 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add98dcc-57a5-42c9-b910-b7633c28d043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:35:09.190515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add98dcc-57a5-42c9-b910-b7633c28d043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.192103 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add98dcc-57a5-42c9-b910-b7633c28d043 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.203856 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1418/2838] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:35:09.208540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacef86-8c18-4b48-9807-346c78564f1f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.219083 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacef86-8c18-4b48-9807-346c78564f1f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.223926 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac778cf-5e73-4eb4-a5a7-77f362fc634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.224561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac778cf-5e73-4eb4-a5a7-77f362fc634d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.224838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac778cf-5e73-4eb4-a5a7-77f362fc634d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.225104 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac778cf-5e73-4eb4-a5a7-77f362fc634d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.225718 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac778cf-5e73-4eb4-a5a7-77f362fc634d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.228866 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac778cf-5e73-4eb4-a5a7-77f362fc634d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.255646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1421/2839] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:35:09 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:35:09.260709 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add98dcc-57a5-42c9-b910-b7633c28d043 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.270502 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add98dcc-57a5-42c9-b910-b7633c28d043 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.327592 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c9ed5-91d8-4406-8100-c1b2f901e41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.328129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c9ed5-91d8-4406-8100-c1b2f901e41f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.328429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c9ed5-91d8-4406-8100-c1b2f901e41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.328707 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c9ed5-91d8-4406-8100-c1b2f901e41f admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.329320 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c9ed5-91d8-4406-8100-c1b2f901e41f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.331227 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c9ed5-91d8-4406-8100-c1b2f901e41f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.352927 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1419/2840] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:35:09 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:35:09.357171 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac778cf-5e73-4eb4-a5a7-77f362fc634d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.367590 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac778cf-5e73-4eb4-a5a7-77f362fc634d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.372289 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0bf88-ce9b-40a9-b655-c997335bff34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.372925 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0bf88-ce9b-40a9-b655-c997335bff34 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.373204 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0bf88-ce9b-40a9-b655-c997335bff34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.373475 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0bf88-ce9b-40a9-b655-c997335bff34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.374118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0bf88-ce9b-40a9-b655-c997335bff34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.375727 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0bf88-ce9b-40a9-b655-c997335bff34 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.386117 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1422/2841] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:35:09.390155 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c9ed5-91d8-4406-8100-c1b2f901e41f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.402052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c9ed5-91d8-4406-8100-c1b2f901e41f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.406908 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba2b56-b384-4e62-a2f4-ae3ee97a631a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.407507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba2b56-b384-4e62-a2f4-ae3ee97a631a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.407786 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba2b56-b384-4e62-a2f4-ae3ee97a631a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.408061 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba2b56-b384-4e62-a2f4-ae3ee97a631a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.408733 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba2b56-b384-4e62-a2f4-ae3ee97a631a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.410373 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba2b56-b384-4e62-a2f4-ae3ee97a631a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.421195 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1420/2842] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:35:09.425217 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0bf88-ce9b-40a9-b655-c997335bff34 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.435395 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0bf88-ce9b-40a9-b655-c997335bff34 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.439971 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb267e7-1f06-485c-a426-0a0c90c2f0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.440647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb267e7-1f06-485c-a426-0a0c90c2f0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.440927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb267e7-1f06-485c-a426-0a0c90c2f0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.441200 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb267e7-1f06-485c-a426-0a0c90c2f0b2 admin admin] PATH_INFO: `/v3/projects/dc1d836794b94f198cbe8059a3d5f38c/users/67b7d4aaec144b24ab927338e23e8713/roles/a7f0f98b4677401690c354299b049892` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.456848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb267e7-1f06-485c-a426-0a0c90c2f0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1d836794b94f198cbe8059a3d5f38c, user_id=67b7d4aaec144b24ab927338e23e8713, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.459516 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb267e7-1f06-485c-a426-0a0c90c2f0b2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.472822 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1423/2843] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:35:09 2022] PUT /identity/v3/projects/dc1d836794b94f198cbe8059a3d5f38c/users/67b7d4aaec144b24ab927338e23e8713/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:09.484038 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbea36f-ffcc-42cb-a621-6d0c87821a6c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.484325 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbea36f-ffcc-42cb-a621-6d0c87821a6c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.484629 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbea36f-ffcc-42cb-a621-6d0c87821a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.511912 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8fbea36f-ffcc-42cb-a621-6d0c87821a6c None None] MFA Rules not processed for user `67b7d4aaec144b24ab927338e23e8713`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:09.513149 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8fbea36f-ffcc-42cb-a621-6d0c87821a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:09.555590 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:09.578165 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1421/2844] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:09.582667 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb267e7-1f06-485c-a426-0a0c90c2f0b2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.643765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb267e7-1f06-485c-a426-0a0c90c2f0b2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.648515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6678e0f-df65-4c8d-bb57-806b1cdf1d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.649203 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6678e0f-df65-4c8d-bb57-806b1cdf1d14 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.649502 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6678e0f-df65-4c8d-bb57-806b1cdf1d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.649770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6678e0f-df65-4c8d-bb57-806b1cdf1d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.650390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6678e0f-df65-4c8d-bb57-806b1cdf1d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.652294 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6678e0f-df65-4c8d-bb57-806b1cdf1d14 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.684006 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1424/2845] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:35:09 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:35:09.687934 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbea36f-ffcc-42cb-a621-6d0c87821a6c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.698750 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbea36f-ffcc-42cb-a621-6d0c87821a6c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.753652 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61387285-9d6a-4003-84b2-6d8c4366fd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.754240 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61387285-9d6a-4003-84b2-6d8c4366fd32 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.754520 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61387285-9d6a-4003-84b2-6d8c4366fd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.754792 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61387285-9d6a-4003-84b2-6d8c4366fd32 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.755420 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61387285-9d6a-4003-84b2-6d8c4366fd32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.757363 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61387285-9d6a-4003-84b2-6d8c4366fd32 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.778396 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1422/2846] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:35:09 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:35:09.782566 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6678e0f-df65-4c8d-bb57-806b1cdf1d14 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.793603 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6678e0f-df65-4c8d-bb57-806b1cdf1d14 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.798213 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04e876-d56d-4b5b-9c05-0f08e54f9ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.798862 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04e876-d56d-4b5b-9c05-0f08e54f9ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.799140 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04e876-d56d-4b5b-9c05-0f08e54f9ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.799412 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04e876-d56d-4b5b-9c05-0f08e54f9ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.800048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04e876-d56d-4b5b-9c05-0f08e54f9ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.801688 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04e876-d56d-4b5b-9c05-0f08e54f9ff7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.812561 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1425/2847] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 15:35:09.816442 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61387285-9d6a-4003-84b2-6d8c4366fd32 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.827260 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61387285-9d6a-4003-84b2-6d8c4366fd32 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.831868 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b05cd-a219-4e50-8562-c302131fbbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.832474 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b05cd-a219-4e50-8562-c302131fbbde admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.832752 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b05cd-a219-4e50-8562-c302131fbbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.833027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b05cd-a219-4e50-8562-c302131fbbde admin admin] PATH_INFO: `/v3/projects/ef6114084c2f4f419355aabeeaf50d91/users/0c6f0b30c7ea42419b1b1e5c32519707/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.848245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b05cd-a219-4e50-8562-c302131fbbde admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6114084c2f4f419355aabeeaf50d91, user_id=0c6f0b30c7ea42419b1b1e5c32519707, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.850936 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b05cd-a219-4e50-8562-c302131fbbde admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.864850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1423/2848] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:35:09 2022] PUT /identity/v3/projects/ef6114084c2f4f419355aabeeaf50d91/users/0c6f0b30c7ea42419b1b1e5c32519707/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:09.869281 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04e876-d56d-4b5b-9c05-0f08e54f9ff7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.880417 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04e876-d56d-4b5b-9c05-0f08e54f9ff7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.938578 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92425e0f-91b8-4cea-ae44-3f7082121554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.939188 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92425e0f-91b8-4cea-ae44-3f7082121554 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.939470 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92425e0f-91b8-4cea-ae44-3f7082121554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.939747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92425e0f-91b8-4cea-ae44-3f7082121554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.940404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92425e0f-91b8-4cea-ae44-3f7082121554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.942036 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92425e0f-91b8-4cea-ae44-3f7082121554 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.951725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1426/2849] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:35:09.955897 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b05cd-a219-4e50-8562-c302131fbbde admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:09.966892 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b05cd-a219-4e50-8562-c302131fbbde admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:09.971490 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b45ea-7e12-483d-9936-c5b05e36c381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:09.972182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b45ea-7e12-483d-9936-c5b05e36c381 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.972620 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b45ea-7e12-483d-9936-c5b05e36c381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.973029 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b45ea-7e12-483d-9936-c5b05e36c381 admin admin] PATH_INFO: `/v3/projects/ef6114084c2f4f419355aabeeaf50d91/users/0c6f0b30c7ea42419b1b1e5c32519707/roles/52514e32ecde4219ab61ab4cbe90a063` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:09.975237 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b45ea-7e12-483d-9936-c5b05e36c381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6114084c2f4f419355aabeeaf50d91, user_id=0c6f0b30c7ea42419b1b1e5c32519707, role_id=52514e32ecde4219ab61ab4cbe90a063)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:09.978763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b45ea-7e12-483d-9936-c5b05e36c381 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:09.989070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1424/2850] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:35:09 2022] PUT /identity/v3/projects/ef6114084c2f4f419355aabeeaf50d91/users/0c6f0b30c7ea42419b1b1e5c32519707/roles/52514e32ecde4219ab61ab4cbe90a063 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:09.999238 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35273075-ddb6-4c79-9f1a-bf0d32e4a685 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:09.999531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35273075-ddb6-4c79-9f1a-bf0d32e4a685 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:09.999813 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35273075-ddb6-4c79-9f1a-bf0d32e4a685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:10.020218 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:10.030779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-35273075-ddb6-4c79-9f1a-bf0d32e4a685 None None] MFA Rules not processed for user `0c6f0b30c7ea42419b1b1e5c32519707`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:10.032299 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35273075-ddb6-4c79-9f1a-bf0d32e4a685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:10.105327 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1427/2851] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:10.116124 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b45ea-7e12-483d-9936-c5b05e36c381 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:10.118666 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-011b45ea-7e12-483d-9936-c5b05e36c381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:10.175931 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b45ea-7e12-483d-9936-c5b05e36c381 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:10.179761 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a325c7-83f5-45e9-aa9e-fb130e48fa6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:10.181070 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a325c7-83f5-45e9-aa9e-fb130e48fa6e service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:10.181350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a325c7-83f5-45e9-aa9e-fb130e48fa6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:10.181618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a325c7-83f5-45e9-aa9e-fb130e48fa6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:10.228776 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:10.238719 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a325c7-83f5-45e9-aa9e-fb130e48fa6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:10.240167 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a325c7-83f5-45e9-aa9e-fb130e48fa6e service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:10.249654 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1425/2852] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:12.781947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35273075-ddb6-4c79-9f1a-bf0d32e4a685 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:12.843911 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35273075-ddb6-4c79-9f1a-bf0d32e4a685 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:12.848442 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:12.849066 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:12.849345 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:12.849619 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] PATH_INFO: `/v3/users/67b7d4aaec144b24ab927338e23e8713` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:12.850753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67b7d4aaec144b24ab927338e23e8713)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:12.852889 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:12.899745 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67b7d4aaec144b24ab927338e23e8713'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:12.900375 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67b7d4aaec144b24ab927338e23e8713'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:12.901647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67b7d4aaec144b24ab927338e23e8713'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:12.908504 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67b7d4aaec144b24ab927338e23e8713'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:12.920994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67b7d4aaec144b24ab927338e23e8713'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:12.934706 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1428/2853] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:35:12 2022] DELETE /identity/v3/users/67b7d4aaec144b24ab927338e23e8713 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:12.939131 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c1ac2-33e9-4ef5-8ff4-bb508ff05786 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:12.939594 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c1ac2-33e9-4ef5-8ff4-bb508ff05786 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:12.939714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c1ac2-33e9-4ef5-8ff4-bb508ff05786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:12.958424 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-025c1ac2-33e9-4ef5-8ff4-bb508ff05786 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:12.959604 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-025c1ac2-33e9-4ef5-8ff4-bb508ff05786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:13.020259 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1426/2854] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:13.030709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:13.032103 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:13.046817 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:13.092920 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bffcf-2134-453c-bf1c-232a7ba87f69 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:13.096639 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a1152-56d1-4813-81fb-888059e9c882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:13.097259 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a1152-56d1-4813-81fb-888059e9c882 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:13.097565 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a1152-56d1-4813-81fb-888059e9c882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:13.097838 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a1152-56d1-4813-81fb-888059e9c882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:13.107046 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a1152-56d1-4813-81fb-888059e9c882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:13.108520 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a1152-56d1-4813-81fb-888059e9c882 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:13.117773 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1429/2855] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:13.392571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c1ac2-33e9-4ef5-8ff4-bb508ff05786 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:13.394052 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-025c1ac2-33e9-4ef5-8ff4-bb508ff05786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:13.407332 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c1ac2-33e9-4ef5-8ff4-bb508ff05786 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:13.412464 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:13.413348 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:13.413632 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:13.413903 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] PATH_INFO: `/v3/projects/dc1d836794b94f198cbe8059a3d5f38c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:13.415036 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc1d836794b94f198cbe8059a3d5f38c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:13.417006 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:13.470702 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc1d836794b94f198cbe8059a3d5f38c', 'request_id': 'req-53cee444-3250-4975-82f4-123b87bb026b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.476035 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc1d836794b94f198cbe8059a3d5f38c', 'request_id': 'req-53cee444-3250-4975-82f4-123b87bb026b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.482667 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc1d836794b94f198cbe8059a3d5f38c', 'request_id': 'req-53cee444-3250-4975-82f4-123b87bb026b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.493694 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] The token cache is being invalidate because project dc1d836794b94f198cbe8059a3d5f38c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:35:13.494004 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.494679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.496778 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1427/2856] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:35:13 2022] DELETE /identity/v3/projects/dc1d836794b94f198cbe8059a3d5f38c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:13.500757 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a1152-56d1-4813-81fb-888059e9c882 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:13.510456 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a1152-56d1-4813-81fb-888059e9c882 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:13.567220 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:13.567851 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:13.568134 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:13.568436 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] PATH_INFO: `/v3/users/0c6f0b30c7ea42419b1b1e5c32519707` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:13.569558 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c6f0b30c7ea42419b1b1e5c32519707)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:13.571503 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:13.616881 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c6f0b30c7ea42419b1b1e5c32519707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.617515 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c6f0b30c7ea42419b1b1e5c32519707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.618124 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c6f0b30c7ea42419b1b1e5c32519707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.625206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c6f0b30c7ea42419b1b1e5c32519707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.636978 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-96b42af9-b895-4f76-a104-4e18308af76d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c6f0b30c7ea42419b1b1e5c32519707'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:13.649693 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1430/2857] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:35:13 2022] DELETE /identity/v3/users/0c6f0b30c7ea42419b1b1e5c32519707 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:13.924710 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:13.926255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:13.940898 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cee444-3250-4975-82f4-123b87bb026b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:13.998027 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:13.998575 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:13.998855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:13.999125 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] PATH_INFO: `/v3/projects/ef6114084c2f4f419355aabeeaf50d91` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.000223 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6114084c2f4f419355aabeeaf50d91)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:14.002174 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:14.019015 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:14.050783 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6114084c2f4f419355aabeeaf50d91', 'request_id': 'req-f6d458de-2fb0-4d0a-b015-44f1f7636636'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:14.056673 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6114084c2f4f419355aabeeaf50d91', 'request_id': 'req-f6d458de-2fb0-4d0a-b015-44f1f7636636'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:14.064787 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6114084c2f4f419355aabeeaf50d91', 'request_id': 'req-f6d458de-2fb0-4d0a-b015-44f1f7636636'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:14.076419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] The token cache is being invalidate because project ef6114084c2f4f419355aabeeaf50d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:35:14.076718 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:14.077350 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f6d458de-2fb0-4d0a-b015-44f1f7636636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:14.078836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1428/2858] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:35:13 2022] DELETE /identity/v3/projects/ef6114084c2f4f419355aabeeaf50d91 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:14.083793 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd977e-0905-4efa-9269-bde62ba5c0b7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.084284 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd977e-0905-4efa-9269-bde62ba5c0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.084421 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd977e-0905-4efa-9269-bde62ba5c0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.104576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4fbd977e-0905-4efa-9269-bde62ba5c0b7 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:14.105947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4fbd977e-0905-4efa-9269-bde62ba5c0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:14.134997 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:14.167378 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1431/2859] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:14.175280 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8eb79b-54c3-43f2-a7cb-21fdd8a0b812 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.175571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8eb79b-54c3-43f2-a7cb-21fdd8a0b812 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.175853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8eb79b-54c3-43f2-a7cb-21fdd8a0b812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.195885 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ad8eb79b-54c3-43f2-a7cb-21fdd8a0b812 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:14.197190 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ad8eb79b-54c3-43f2-a7cb-21fdd8a0b812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:14.205780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1429/2860] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:14.211489 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd977e-0905-4efa-9269-bde62ba5c0b7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:14.221269 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd977e-0905-4efa-9269-bde62ba5c0b7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:14.225754 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648f062-232a-46f2-a771-9d84f6dd332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:14.226341 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648f062-232a-46f2-a771-9d84f6dd332a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.226617 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648f062-232a-46f2-a771-9d84f6dd332a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.226887 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648f062-232a-46f2-a771-9d84f6dd332a admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.227576 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648f062-232a-46f2-a771-9d84f6dd332a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:35:14.227865 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648f062-232a-46f2-a771-9d84f6dd332a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:14.229675 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b648f062-232a-46f2-a771-9d84f6dd332a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:14.241004 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1432/2861] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:35:14.245836 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8eb79b-54c3-43f2-a7cb-21fdd8a0b812 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:14.255023 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8eb79b-54c3-43f2-a7cb-21fdd8a0b812 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:14.259687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd60a0a-9e22-4d4f-947c-3ad640894ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:14.260273 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd60a0a-9e22-4d4f-947c-3ad640894ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.261170 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd60a0a-9e22-4d4f-947c-3ad640894ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.261447 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd60a0a-9e22-4d4f-947c-3ad640894ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.262078 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd60a0a-9e22-4d4f-947c-3ad640894ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:14.264030 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd60a0a-9e22-4d4f-947c-3ad640894ef6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:14.290162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1430/2862] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:35:14 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:35:14.294226 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648f062-232a-46f2-a771-9d84f6dd332a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:14.304057 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648f062-232a-46f2-a771-9d84f6dd332a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:14.364803 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ef00f-ada1-45bb-8745-25158cc160d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:14.365415 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ef00f-ada1-45bb-8745-25158cc160d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.365696 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ef00f-ada1-45bb-8745-25158cc160d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.365972 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ef00f-ada1-45bb-8745-25158cc160d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.366610 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ef00f-ada1-45bb-8745-25158cc160d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:14.368690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ef00f-ada1-45bb-8745-25158cc160d9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:14.391183 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1433/2863] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:35:14 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 15:35:14.395228 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd60a0a-9e22-4d4f-947c-3ad640894ef6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:14.405432 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd60a0a-9e22-4d4f-947c-3ad640894ef6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:14.409932 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35756f78-165e-45b9-b42f-e215a58ce489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:14.410515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756f78-165e-45b9-b42f-e215a58ce489 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.410793 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756f78-165e-45b9-b42f-e215a58ce489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.411063 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756f78-165e-45b9-b42f-e215a58ce489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.411695 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35756f78-165e-45b9-b42f-e215a58ce489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:14.413335 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35756f78-165e-45b9-b42f-e215a58ce489 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:14.423245 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1431/2864] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:35:14.427111 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ef00f-ada1-45bb-8745-25158cc160d9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:14.439102 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ef00f-ada1-45bb-8745-25158cc160d9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:14.443906 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a9968-046b-4581-a5cc-1b0f87c2f81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:14.444537 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a9968-046b-4581-a5cc-1b0f87c2f81d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.444814 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a9968-046b-4581-a5cc-1b0f87c2f81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.445080 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a9968-046b-4581-a5cc-1b0f87c2f81d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.445739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a9968-046b-4581-a5cc-1b0f87c2f81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:14.447338 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a9968-046b-4581-a5cc-1b0f87c2f81d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:14.458801 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1434/2865] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:35:14.463115 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35756f78-165e-45b9-b42f-e215a58ce489 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:14.473133 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35756f78-165e-45b9-b42f-e215a58ce489 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:14.477741 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221152c7-1c92-4237-9c88-388787eb64f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:14.478384 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221152c7-1c92-4237-9c88-388787eb64f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.478663 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221152c7-1c92-4237-9c88-388787eb64f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.478935 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221152c7-1c92-4237-9c88-388787eb64f0 admin admin] PATH_INFO: `/v3/projects/216d9bc1f0ee480a9312577f87989937/users/abdc2b30b3634e91967afa52b71846c0/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.493236 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221152c7-1c92-4237-9c88-388787eb64f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216d9bc1f0ee480a9312577f87989937, user_id=abdc2b30b3634e91967afa52b71846c0, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:14.496445 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221152c7-1c92-4237-9c88-388787eb64f0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:14.508899 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1432/2866] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:35:14 2022] PUT /identity/v3/projects/216d9bc1f0ee480a9312577f87989937/users/abdc2b30b3634e91967afa52b71846c0/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:14.519540 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec56a8-ec2a-412c-ab9f-ee85e56b3755 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.519823 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec56a8-ec2a-412c-ab9f-ee85e56b3755 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.520094 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec56a8-ec2a-412c-ab9f-ee85e56b3755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.548048 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f3ec56a8-ec2a-412c-ab9f-ee85e56b3755 None None] MFA Rules not processed for user `abdc2b30b3634e91967afa52b71846c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:14.549293 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f3ec56a8-ec2a-412c-ab9f-ee85e56b3755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:14.614733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1435/2867] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:14.625846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221152c7-1c92-4237-9c88-388787eb64f0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:14.627511 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-221152c7-1c92-4237-9c88-388787eb64f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:14.685784 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221152c7-1c92-4237-9c88-388787eb64f0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:14.689593 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9f9e5-1712-4af9-8cc3-47c4f2a2cf8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:14.690191 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9f9e5-1712-4af9-8cc3-47c4f2a2cf8c service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:14.690467 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9f9e5-1712-4af9-8cc3-47c4f2a2cf8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:14.690736 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9f9e5-1712-4af9-8cc3-47c4f2a2cf8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:14.699176 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9f9e5-1712-4af9-8cc3-47c4f2a2cf8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:14.700723 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9f9e5-1712-4af9-8cc3-47c4f2a2cf8c service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:14.709302 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1433/2868] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:17.705916 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec56a8-ec2a-412c-ab9f-ee85e56b3755 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:17.863607 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:17.897404 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec56a8-ec2a-412c-ab9f-ee85e56b3755 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:17.903649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:17.904287 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:17.904596 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:17.904871 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] PATH_INFO: `/v3/users/abdc2b30b3634e91967afa52b71846c0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:17.906008 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abdc2b30b3634e91967afa52b71846c0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:17.907956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:17.956739 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abdc2b30b3634e91967afa52b71846c0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:17.957286 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abdc2b30b3634e91967afa52b71846c0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:17.957900 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abdc2b30b3634e91967afa52b71846c0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:17.965002 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abdc2b30b3634e91967afa52b71846c0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:17.977108 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abdc2b30b3634e91967afa52b71846c0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:17.990270 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1436/2869] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:35:17 2022] DELETE /identity/v3/users/abdc2b30b3634e91967afa52b71846c0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:17.995146 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc6e84-77eb-42f0-826f-9dda00770a4c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:17.995979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc6e84-77eb-42f0-826f-9dda00770a4c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:17.995979 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc6e84-77eb-42f0-826f-9dda00770a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:18.015861 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a8bc6e84-77eb-42f0-826f-9dda00770a4c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:18.017097 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a8bc6e84-77eb-42f0-826f-9dda00770a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:18.059419 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:18.079981 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1434/2870] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:18.089897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:18.091310 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:18.147848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c3573-365f-422b-a576-6064ef48242f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:18.151657 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0df49b-1394-43b7-9b16-82bf309cbfd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:18.152252 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0df49b-1394-43b7-9b16-82bf309cbfd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:18.152598 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0df49b-1394-43b7-9b16-82bf309cbfd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:18.152875 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0df49b-1394-43b7-9b16-82bf309cbfd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:18.161936 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0df49b-1394-43b7-9b16-82bf309cbfd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:18.163381 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0df49b-1394-43b7-9b16-82bf309cbfd3 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:18.172902 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1437/2871] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:18.683409 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc6e84-77eb-42f0-826f-9dda00770a4c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:18.685053 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a8bc6e84-77eb-42f0-826f-9dda00770a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:18.697109 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc6e84-77eb-42f0-826f-9dda00770a4c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:18.702392 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:18.703051 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:18.703327 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:18.703747 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] PATH_INFO: `/v3/projects/216d9bc1f0ee480a9312577f87989937` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:18.704906 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=216d9bc1f0ee480a9312577f87989937)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:18.706826 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:18.754171 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '216d9bc1f0ee480a9312577f87989937', 'request_id': 'req-3c356ace-629a-4e3a-990e-8c851ee98f58'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:18.759760 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '216d9bc1f0ee480a9312577f87989937', 'request_id': 'req-3c356ace-629a-4e3a-990e-8c851ee98f58'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:18.767049 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '216d9bc1f0ee480a9312577f87989937', 'request_id': 'req-3c356ace-629a-4e3a-990e-8c851ee98f58'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:18.778569 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] The token cache is being invalidate because project 216d9bc1f0ee480a9312577f87989937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:35:18.778850 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:18.779494 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3c356ace-629a-4e3a-990e-8c851ee98f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:18.780984 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1435/2872] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:35:18 2022] DELETE /identity/v3/projects/216d9bc1f0ee480a9312577f87989937 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:18.785765 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423a86-a40b-4c12-b1c4-05c402aecfbc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:18.786247 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423a86-a40b-4c12-b1c4-05c402aecfbc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:18.786448 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423a86-a40b-4c12-b1c4-05c402aecfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:18.806394 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2c423a86-a40b-4c12-b1c4-05c402aecfbc None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:18.807770 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2c423a86-a40b-4c12-b1c4-05c402aecfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:18.853390 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:18.867725 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1438/2873] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:18.875743 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184d7b-93d1-4b28-88a6-4b22ca1914a7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:18.876026 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184d7b-93d1-4b28-88a6-4b22ca1914a7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:18.876297 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184d7b-93d1-4b28-88a6-4b22ca1914a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:18.895876 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-01184d7b-93d1-4b28-88a6-4b22ca1914a7 None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:18.897095 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-01184d7b-93d1-4b28-88a6-4b22ca1914a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:18.906132 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1436/2874] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:18.910684 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423a86-a40b-4c12-b1c4-05c402aecfbc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:18.920874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423a86-a40b-4c12-b1c4-05c402aecfbc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:18.925356 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491d127-d101-4253-ba31-9103391165f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:18.925947 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491d127-d101-4253-ba31-9103391165f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:18.926221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491d127-d101-4253-ba31-9103391165f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:18.926490 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491d127-d101-4253-ba31-9103391165f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:18.927132 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491d127-d101-4253-ba31-9103391165f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 15:35:18.927427 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491d127-d101-4253-ba31-9103391165f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:18.929304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491d127-d101-4253-ba31-9103391165f2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:18.941130 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1439/2875] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Aug 19 15:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 15:35:18.945358 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01184d7b-93d1-4b28-88a6-4b22ca1914a7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:18.954503 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01184d7b-93d1-4b28-88a6-4b22ca1914a7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:18.959162 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109bd5e-e014-4a55-8d42-b0534cc47a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:18.959772 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109bd5e-e014-4a55-8d42-b0534cc47a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:18.960045 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109bd5e-e014-4a55-8d42-b0534cc47a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:18.960309 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109bd5e-e014-4a55-8d42-b0534cc47a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:18.960957 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109bd5e-e014-4a55-8d42-b0534cc47a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:18.962838 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109bd5e-e014-4a55-8d42-b0534cc47a5c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:18.990362 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1437/2876] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Aug 19 15:35:18 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:35:18.994632 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491d127-d101-4253-ba31-9103391165f2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:19.004660 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491d127-d101-4253-ba31-9103391165f2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:19.061538 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6508c-25e5-409e-ad12-21452770684f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:19.062062 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6508c-25e5-409e-ad12-21452770684f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:19.062343 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6508c-25e5-409e-ad12-21452770684f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:19.062618 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6508c-25e5-409e-ad12-21452770684f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:19.063262 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6508c-25e5-409e-ad12-21452770684f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:19.065206 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf6508c-25e5-409e-ad12-21452770684f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:19.085988 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1440/2877] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Aug 19 15:35:18 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 15:35:19.090763 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109bd5e-e014-4a55-8d42-b0534cc47a5c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:19.100923 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109bd5e-e014-4a55-8d42-b0534cc47a5c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:19.106637 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a388e3a-6ece-40e7-b1d9-e1460e2a70c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:19.107226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388e3a-6ece-40e7-b1d9-e1460e2a70c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:19.107521 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388e3a-6ece-40e7-b1d9-e1460e2a70c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:19.107780 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388e3a-6ece-40e7-b1d9-e1460e2a70c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:19.108444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a388e3a-6ece-40e7-b1d9-e1460e2a70c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:19.110043 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a388e3a-6ece-40e7-b1d9-e1460e2a70c5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:19.120111 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1438/2878] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:35:19.123756 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6508c-25e5-409e-ad12-21452770684f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:19.133779 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6508c-25e5-409e-ad12-21452770684f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:19.138991 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7246b30-ff17-40a1-9d3e-44742b5b4779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:19.139583 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7246b30-ff17-40a1-9d3e-44742b5b4779 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:19.139859 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7246b30-ff17-40a1-9d3e-44742b5b4779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:19.140128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7246b30-ff17-40a1-9d3e-44742b5b4779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:19.140796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7246b30-ff17-40a1-9d3e-44742b5b4779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:19.142531 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7246b30-ff17-40a1-9d3e-44742b5b4779 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:19.153518 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1441/2879] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Aug 19 15:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 15:35:19.157687 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a388e3a-6ece-40e7-b1d9-e1460e2a70c5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:19.167618 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a388e3a-6ece-40e7-b1d9-e1460e2a70c5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:19.172182 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94391105-8465-4c81-ba00-b6607effa908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:19.172853 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94391105-8465-4c81-ba00-b6607effa908 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:19.173129 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94391105-8465-4c81-ba00-b6607effa908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:19.173401 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94391105-8465-4c81-ba00-b6607effa908 admin admin] PATH_INFO: `/v3/projects/2558a7c80abd410bafa20a3aadc0e74d/users/72f1d4cb0e89482f916d6ee9bfe9bcb1/roles/a7f0f98b4677401690c354299b049892` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:19.187770 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94391105-8465-4c81-ba00-b6607effa908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2558a7c80abd410bafa20a3aadc0e74d, user_id=72f1d4cb0e89482f916d6ee9bfe9bcb1, role_id=a7f0f98b4677401690c354299b049892)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:19.190469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94391105-8465-4c81-ba00-b6607effa908 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:19.203513 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1439/2880] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Aug 19 15:35:19 2022] PUT /identity/v3/projects/2558a7c80abd410bafa20a3aadc0e74d/users/72f1d4cb0e89482f916d6ee9bfe9bcb1/roles/a7f0f98b4677401690c354299b049892 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:19.215350 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08788628-2075-4bb3-b145-5d8f0996a566 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:19.215658 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08788628-2075-4bb3-b145-5d8f0996a566 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:19.215956 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08788628-2075-4bb3-b145-5d8f0996a566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:19.243232 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-08788628-2075-4bb3-b145-5d8f0996a566 None None] MFA Rules not processed for user `72f1d4cb0e89482f916d6ee9bfe9bcb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:19.244461 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-08788628-2075-4bb3-b145-5d8f0996a566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:19.309646 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1442/2881] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:19.320093 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94391105-8465-4c81-ba00-b6607effa908 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:19.321797 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-94391105-8465-4c81-ba00-b6607effa908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:19.379199 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94391105-8465-4c81-ba00-b6607effa908 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:19.383086 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af5034-e04c-4397-994a-79657dd77c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6b4d6a93c7e49009aca527e281b5c3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:19.383689 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af5034-e04c-4397-994a-79657dd77c0c service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:19.383971 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af5034-e04c-4397-994a-79657dd77c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:19.384246 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af5034-e04c-4397-994a-79657dd77c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:19.394068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af5034-e04c-4397-994a-79657dd77c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:19.395606 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af5034-e04c-4397-994a-79657dd77c0c service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:19.405493 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1440/2882] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:20.757304 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b253fa-a695-41bc-92bc-0e0e91778302 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:20.757796 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b253fa-a695-41bc-92bc-0e0e91778302 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:20.757915 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b253fa-a695-41bc-92bc-0e0e91778302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:20.778300 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d0b253fa-a695-41bc-92bc-0e0e91778302 None None] MFA Rules not processed for user `72f1d4cb0e89482f916d6ee9bfe9bcb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:20.779414 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d0b253fa-a695-41bc-92bc-0e0e91778302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:20.786513 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1443/2883] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:29.400725 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:30.771652 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:35.623074 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec06a9-ce3a-48a8-b82f-db8a1d24cb91 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:35.623581 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec06a9-ce3a-48a8-b82f-db8a1d24cb91 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:35.623697 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec06a9-ce3a-48a8-b82f-db8a1d24cb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:35.643072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-10ec06a9-ce3a-48a8-b82f-db8a1d24cb91 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:35.644247 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-10ec06a9-ce3a-48a8-b82f-db8a1d24cb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:35.699094 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1441/2884] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:35:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:35.745721 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b253fa-a695-41bc-92bc-0e0e91778302 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:35.747180 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d0b253fa-a695-41bc-92bc-0e0e91778302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:35.829339 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b253fa-a695-41bc-92bc-0e0e91778302 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:35.834301 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec5685-3d09-48bf-b903-ea43e6023b11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:35.834930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec5685-3d09-48bf-b903-ea43e6023b11 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:35.835209 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec5685-3d09-48bf-b903-ea43e6023b11 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:35.835480 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec5685-3d09-48bf-b903-ea43e6023b11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:35.844239 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec5685-3d09-48bf-b903-ea43e6023b11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:35.845855 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec5685-3d09-48bf-b903-ea43e6023b11 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:35.853897 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1444/2885] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:35:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:35:36.367288 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1c77a-9da6-48d3-9634-1739ad983387 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:36.367582 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1c77a-9da6-48d3-9634-1739ad983387 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:36.367864 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1c77a-9da6-48d3-9634-1739ad983387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:36.391902 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-30c1c77a-9da6-48d3-9634-1739ad983387 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:36.393256 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-30c1c77a-9da6-48d3-9634-1739ad983387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:36.401456 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1442/2886] 10.222.0.75 () {58 vars in 967 bytes} [Fri Aug 19 15:35:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:36.415318 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec5685-3d09-48bf-b903-ea43e6023b11 service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:36.427669 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec5685-3d09-48bf-b903-ea43e6023b11 service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:36.440040 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4236886-1c47-453e-a225-89a6ce58c461 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:36.441003 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236886-1c47-453e-a225-89a6ce58c461 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:36.441529 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236886-1c47-453e-a225-89a6ce58c461 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:36.442047 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236886-1c47-453e-a225-89a6ce58c461 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:36.460521 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4236886-1c47-453e-a225-89a6ce58c461 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:36.462128 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4236886-1c47-453e-a225-89a6ce58c461 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:36.471749 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1445/2887] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:35:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:35:36.703992 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557de45-702b-45c5-ab1c-83747ed991f4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:36.704574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557de45-702b-45c5-ab1c-83747ed991f4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:36.704574 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557de45-702b-45c5-ab1c-83747ed991f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:36.724213 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7557de45-702b-45c5-ab1c-83747ed991f4 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:36.725444 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7557de45-702b-45c5-ab1c-83747ed991f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:36.733379 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1443/2888] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Aug 19 15:35:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:36.745429 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4236886-1c47-453e-a225-89a6ce58c461 service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:36.755994 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4236886-1c47-453e-a225-89a6ce58c461 service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:36.761205 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730aebf4-fba1-4bee-8c01-a9e7a9e96add service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:36.761930 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aebf4-fba1-4bee-8c01-a9e7a9e96add service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:36.762393 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aebf4-fba1-4bee-8c01-a9e7a9e96add service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:36.762753 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aebf4-fba1-4bee-8c01-a9e7a9e96add service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:36.771314 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730aebf4-fba1-4bee-8c01-a9e7a9e96add service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:36.773065 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730aebf4-fba1-4bee-8c01-a9e7a9e96add service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:36.781852 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1446/2889] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:35:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:35:37.116380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a7572-3fb0-4b92-a317-1f0b3db15462 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:37.116696 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a7572-3fb0-4b92-a317-1f0b3db15462 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:37.116970 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a7572-3fb0-4b92-a317-1f0b3db15462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:37.138253 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-362a7572-3fb0-4b92-a317-1f0b3db15462 None None] MFA Rules not processed for user `4d4a71b6005343cda93a92465067a7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:37.139533 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-362a7572-3fb0-4b92-a317-1f0b3db15462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:37.147291 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1444/2890] 10.222.0.75 () {58 vars in 967 bytes} [Fri Aug 19 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 15:35:37.160712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730aebf4-fba1-4bee-8c01-a9e7a9e96add service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:37.171586 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730aebf4-fba1-4bee-8c01-a9e7a9e96add service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:37.177825 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada6d04-c202-427f-877c-5c628ba7d06d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '383ff491387642a3ab45ab692f94189c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:37.178418 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada6d04-c202-427f-877c-5c628ba7d06d service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:37.178690 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada6d04-c202-427f-877c-5c628ba7d06d service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:37.178979 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada6d04-c202-427f-877c-5c628ba7d06d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:37.187854 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada6d04-c202-427f-877c-5c628ba7d06d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:37.189539 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada6d04-c202-427f-877c-5c628ba7d06d service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:37.198118 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1447/2891] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Aug 19 15:35:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 15:35:39.595009 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a7572-3fb0-4b92-a317-1f0b3db15462 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:39.673675 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a7572-3fb0-4b92-a317-1f0b3db15462 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:39.678515 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:39.679427 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:39.679817 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:39.680189 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] PATH_INFO: `/v3/users/72f1d4cb0e89482f916d6ee9bfe9bcb1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:39.681469 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f1d4cb0e89482f916d6ee9bfe9bcb1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:39.683616 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:39.748625 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f1d4cb0e89482f916d6ee9bfe9bcb1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:39.756512 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f1d4cb0e89482f916d6ee9bfe9bcb1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:39.757478 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f1d4cb0e89482f916d6ee9bfe9bcb1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:39.770116 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f1d4cb0e89482f916d6ee9bfe9bcb1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:39.777867 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f1d4cb0e89482f916d6ee9bfe9bcb1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:39.787679 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1445/2892] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:35:39 2022] DELETE /identity/v3/users/72f1d4cb0e89482f916d6ee9bfe9bcb1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:39.789649 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aeb75-1057-4053-b67e-1795c245e82c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:39.790167 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aeb75-1057-4053-b67e-1795c245e82c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:39.790373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8aeb75-1057-4053-b67e-1795c245e82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:39.810927 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fb8aeb75-1057-4053-b67e-1795c245e82c None None] MFA Rules not processed for user `0ba95fb65871415ca38c574ddd6396b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 15:35:39.812119 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fb8aeb75-1057-4053-b67e-1795c245e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:39.872295 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1448/2893] 10.222.0.75 () {58 vars in 958 bytes} [Fri Aug 19 15:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 15:35:39.881798 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:39.883281 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:39.891072 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:39.946380 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bc46f-9eec-4c1b-95fe-6cda0a68ca3b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:39.950567 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaee56e-e6e1-49de-86c4-8952cc268b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8704aae85084d41bdedb5537ef96e44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:39.951255 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaee56e-e6e1-49de-86c4-8952cc268b5a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:39.951571 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaee56e-e6e1-49de-86c4-8952cc268b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:39.951846 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaee56e-e6e1-49de-86c4-8952cc268b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:39.961777 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaee56e-e6e1-49de-86c4-8952cc268b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:39.963226 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaee56e-e6e1-49de-86c4-8952cc268b5a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:39.973907 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1446/2894] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:35:40.154709 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aeb75-1057-4053-b67e-1795c245e82c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:40.156647 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fb8aeb75-1057-4053-b67e-1795c245e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:40.171528 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8aeb75-1057-4053-b67e-1795c245e82c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:40.176545 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:40.177372 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:40.177832 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:40.178767 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] PATH_INFO: `/v3/projects/2558a7c80abd410bafa20a3aadc0e74d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:40.180712 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2558a7c80abd410bafa20a3aadc0e74d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:40.182769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:40.222769 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:40.233584 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2558a7c80abd410bafa20a3aadc0e74d', 'request_id': 'req-4eb7819c-b48f-4beb-8686-b16ba5b7f526'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:40.239382 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2558a7c80abd410bafa20a3aadc0e74d', 'request_id': 'req-4eb7819c-b48f-4beb-8686-b16ba5b7f526'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:40.247874 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2558a7c80abd410bafa20a3aadc0e74d', 'request_id': 'req-4eb7819c-b48f-4beb-8686-b16ba5b7f526'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:40.259788 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] The token cache is being invalidate because project 2558a7c80abd410bafa20a3aadc0e74d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:35:40.260073 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:40.260747 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:40.262221 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1449/2895] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:35:40 2022] DELETE /identity/v3/projects/2558a7c80abd410bafa20a3aadc0e74d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:42.715939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaee56e-e6e1-49de-86c4-8952cc268b5a service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:42.715939 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6aaee56e-e6e1-49de-86c4-8952cc268b5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:42.731871 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaee56e-e6e1-49de-86c4-8952cc268b5a service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:42.787559 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:42.788206 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:42.788510 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:42.788789 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] PATH_INFO: `/v3/users/55a7b30c3c094a8088c370a0d79ac98a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:42.789909 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a7b30c3c094a8088c370a0d79ac98a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:42.791893 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:42.839626 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a7b30c3c094a8088c370a0d79ac98a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:42.846397 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a7b30c3c094a8088c370a0d79ac98a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:42.847060 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a7b30c3c094a8088c370a0d79ac98a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:42.859242 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a7b30c3c094a8088c370a0d79ac98a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:42.866685 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a7b30c3c094a8088c370a0d79ac98a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:42.873136 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1447/2896] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:35:42 2022] DELETE /identity/v3/users/55a7b30c3c094a8088c370a0d79ac98a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:43.040954 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:43.042451 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:43.058248 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7819c-b48f-4beb-8686-b16ba5b7f526 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:43.117017 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:43.117673 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:43.117955 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:43.118229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] PATH_INFO: `/v3/projects/7111b041a72b4481b3c8a286589e0a4e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:43.119346 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7111b041a72b4481b3c8a286589e0a4e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:43.121321 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:43.167113 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7111b041a72b4481b3c8a286589e0a4e', 'request_id': 'req-aa30e5e1-789a-45b7-920e-0b63f247b148'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.173517 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7111b041a72b4481b3c8a286589e0a4e', 'request_id': 'req-aa30e5e1-789a-45b7-920e-0b63f247b148'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.180848 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7111b041a72b4481b3c8a286589e0a4e', 'request_id': 'req-aa30e5e1-789a-45b7-920e-0b63f247b148'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.191794 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] The token cache is being invalidate because project 7111b041a72b4481b3c8a286589e0a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:35:43.192106 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.192761 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.194229 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1450/2897] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:35:43 2022] DELETE /identity/v3/projects/7111b041a72b4481b3c8a286589e0a4e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:43.198507 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:43.199949 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:43.212714 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac410fc-9452-4243-9e48-69262e7e956d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:43.238748 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:43.272229 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:43.272882 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:43.273160 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:43.273429 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] PATH_INFO: `/v3/users/686f6293fd7248c59cc05475215b2f30` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:43.274535 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=686f6293fd7248c59cc05475215b2f30)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:43.276484 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:43.323058 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '686f6293fd7248c59cc05475215b2f30'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.330222 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '686f6293fd7248c59cc05475215b2f30'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.330883 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '686f6293fd7248c59cc05475215b2f30'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.343271 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '686f6293fd7248c59cc05475215b2f30'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.350815 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '686f6293fd7248c59cc05475215b2f30'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.357466 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1448/2898] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Aug 19 15:35:43 2022] DELETE /identity/v3/users/686f6293fd7248c59cc05475215b2f30 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 15:35:43.529878 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:35:43.531380 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:35:43.544829 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30e5e1-789a-45b7-920e-0b63f247b148 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:35:43.596290 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:43.608373 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ba95fb65871415ca38c574ddd6396b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9a96491d392440daeaecef9eef8a152', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:35:43.609030 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:35:43.609307 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:35:43.609580 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] PATH_INFO: `/v3/projects/8977fcec1a794f1494a07cc3e0344b68` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:35:43.610708 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8977fcec1a794f1494a07cc3e0344b68)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:35:43.613000 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:35:43.662138 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8977fcec1a794f1494a07cc3e0344b68', 'request_id': 'req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.668347 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8977fcec1a794f1494a07cc3e0344b68', 'request_id': 'req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.677864 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8977fcec1a794f1494a07cc3e0344b68', 'request_id': 'req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.689733 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] The token cache is being invalidate because project 8977fcec1a794f1494a07cc3e0344b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 15:35:43.690097 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.690820 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-73dd4b3e-6063-4dfa-a53f-5ee4dadc0bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 15:35:43.692340 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1451/2899] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Aug 19 15:35:43 2022] DELETE /identity/v3/projects/8977fcec1a794f1494a07cc3e0344b68 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 15:35:53.353413 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:35:53.686656 dv-cinder-835384-20-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 15:36:17.312775 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 15:36:17.314164 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:36:17.384000 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573cf2d-82e7-445f-9eee-0669e57325f6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 15:36:17.441472 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ff231-5908-4bfc-a789-c3c6be9570b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc2920aac4c046aca75473158ec204cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2520016785f144f5be02dae999d3037a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 15:36:17.442068 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ff231-5908-4bfc-a789-c3c6be9570b1 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 15:36:17.442347 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ff231-5908-4bfc-a789-c3c6be9570b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 15:36:17.442619 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ff231-5908-4bfc-a789-c3c6be9570b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 15:36:17.444333 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dd6ff231-5908-4bfc-a789-c3c6be9570b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 15:36:17.455841 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ff231-5908-4bfc-a789-c3c6be9570b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 15:36:17.457451 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ff231-5908-4bfc-a789-c3c6be9570b1 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 15:36:17.467855 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1449/2900] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Aug 19 15:36:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 15:36:27.462945 dv-cinder-835384-20-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=58 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}